Error » Error News and Info » Knowledge Base » SQL - eliminate rows based on duplicate non-key info

Knowledge Base Most common error and how to trouble shoot them off

Post New Thread Reply
  SQL - eliminate rows based on duplicate non-key info
LinkBack Thread Tools Display Modes
Old 09-Feb-2007, 01:01 AM   #1 (permalink)
Administrator
 
Admin's Avatar

Posts: 875
Join Date: Oct 2005
Rep Power: 10 Admin has disabled reputation

IM:
Default SQL - eliminate rows based on duplicate non-key info

Have SQL that builds a view of data that looks like this:
SELECT a.emplid
,a.test_id
,a.ls_data_source
FROM ps_stdnt_test_comp a
, ps_sa_tcmp_rel_tbl b
WHERE a.test_id IN ('GRE','GMAT','TOEFL','CTOFL','TOEFL IBT','PRAX1','PRXC1')
AND a.test_id = b.test_id
AND a.test_component = b.test_component

Result Test scenario looks like this:

Emplid TestId Source
EMPLID01 GRE SELF
EMPLID01 GRE OFF
EMPLID01 TOEFL SELF
EMPLID01 TOEFL OFF

I want to eliminate rows where the empl+testid occurs more than once, and any of the multiple rows has a source of OFF. Basically I want to show self-reported test scores until the official test score comes in, then only show the official row.

Solution:

I would think the subquery would have a much better chance to benefit from proper indexing. On the bad side, the subquery will in fact require an index of some sort to get any type of acceptable performance on a large table.

But for the join, when you add this:

AND aSelf.ls_data_source <> 'OFF'

to the main WHERE clause, I think you reduce the chance of an index being used, I think you reduce the chance of an index being selected.


It will be interesting to see actual results :-)

Although it works with the limited data pulls I cannot TNC's to work when I add the additional fields to pull, I just get all the rows.; probably a quick fix. Here is what I am running (note the 'MAN' instead of 'OFF' - 'MAN is the actual database value). Again; it works until I toss in the fields i have commented out. Scott's works fine and is very quick.

SELECT DISTINCT
aSelf.emplid,
aSelf.test_id,
aSelf.test_component,
COALESCE(aOff.ls_data_source, aSelf.ls_data_source) AS ls_data_source
-- ,aself.score
--,aself.score_letter
-- ,aself.ext_acad_level
-- ,aself.date_loaded
-- ,aself.percentile
-- ,aself.test_admin
-- ,aself.test_index
FROM ps_stdnt_test_comp aSelf
LEFT JOIN ps_stdnt_test_comp aOff ON
aSelf.emplid = aOff.emplid AND
aSelf.test_id = aOff.test_id AND
aSelf.test_component = aOff.test_component AND -- this is the new condition I added . . . add as many of these as is required to build the NK
aOff.ls_data_source = 'MAN'
where aself.emplid = '10947'


SELECT a.emplid
,a.test_id
,a.test_dt
,a.ls_data_source
,b.descrshort
,a.score
,a.score_letter
,a.ext_acad_level
,a.date_loaded
,a.percentile
,a.test_admin
,a.test_index
FROM ps_stdnt_test_comp a
, ps_sa_tcmp_rel_tbl b
WHERE a.test_id IN ('GRE','GMAT','TOEFL','CTOFL','TOEFL IBT','PRAX1','PRXC1')
AND a.test_id = b.test_id
AND a.test_component = b.test_component
AND (a.ls_data_source = 'MAN' OR (NOT EXISTS(SELECT 1 FROM ps_stdnt_test_comp c
WHERE c.emplid = a.emplid AND c.test_id = a.test_id AND c.test_component = a.test_component AND c.ls_data_source = 'MAN')))
Admin is offline  
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Spurl this Post!Reddit!
Reply With Quote
   


   
Post New Thread Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT -8. The time now is 03:34 PM.

Powered by vBulletin® Version 3.7.2
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Search Engine Friendly URLs by vBSEO 3.2.0

DMCA Policy

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227