Debian Codegraph @

docs/source/tutorial/s1_similarity.rst

raw
1 Comparing profiles
2 ^^^^^^^^^^^^^^^^^^^^^^^^^^
3 .. _s1p6:
4
5 It is often useful to compare how similar profiles are, visually and/or statistically.
6 RAW has a dedicated comparison window that allows you to compare residuals, between
7 profiles, ratios between profiles, and use statistical tests (currently only the
8 Correlation Map test is implemented) to check for differences between profiles.
9
10
11 Comparing profiles with residuals
12 ####################################
13
14 The residual is the difference between two profiles. Often we normalize this
15 difference by the uncertainty of one of the profiles, so that large deviations
16 of points with large uncertainties don't dominate the residual. If two profiles
17 agree within the noise level of the data the residual should be flat and
18 randomly distributed. If normalized, you expect most of the residual values to
19 be within +/- 2.5. It is often used when comparing a fit against the data
20 (e.g. comparing the fit of the P(r) transformed to I(q) or of a reconstruction
21 against the data).
22
23 The written version of the tutorial follows.
24
25 #. Clear any data loaded into RAW. Load the **2pol_polymerase.fit** data
26 in the **Tutorial_Data/theory_data/theory_results** folder into the
27 Profiles plot.
28
29 * *Note:* This fit will load in two profiles, experimental data and the
30 CRYSOL fit of a PDB file (high resolution model) to that experimental
31 data. See the :ref:`CRYSOL tutorial <crysol_tutorial>` for how to generate
32 these fits.
33
34 #. Select both the **2pol_polymerase.fit** (experimental data) and
35 **2pol_polymerase_FIT** (theoretical model fit to the data) profiles,
36 right click on one of them and select "Compare Profiles" in the right
37 click menu.
38
39 |compare_residual_right_click_png|
40
41 #. The Compare Profiles window will open. You will see three tabs at the top,
42 the first (and default) shows the residuals between the selected data.
43
44 * *Note:* You can select as many profiles as you want for comparison,
45 not just two.
46
47 * *Tip:* You can use the checkboxes next to the profile names to turn on
48 profiles in the comparison plot. You must have at least two profiles
49 turned on or no residuals will be displayed.
50
51 |compare_residual_main_png|
52
53 #. Use the "Ref. profile" drop down menu to select the **2pol_polymerase_FIT**
54 profile as the reference profile. The reference profile is the profile that
55 is subtracted from every other profile. Here you use the theoretical data
56 as the reference.
57
58 * *Note:* The profile plot will update so that the reference profile is
59 bold and on top of all the other plotted profiles.
60
61 * *Note:* You can see that the residuals have a trend to them, they're
62 above zero at high q (0.15-0.2) and then dip below at mid q (around 0.13).
63 This indicates that the theoretical model, while mostly a good fit, isn't
64 a perfect fit for the experimental data.
65
66 |compare_residual_ref_png|
67
68 #. Use the "Normalize Residuals" checkbox to turn normalization on and off and
69 see what difference it makes for both the overall values and shape of the
70 residual.
71
72 * *Note:* You should see the un-normalized residual becomes more dominated
73 by very low q deviations, but those points are higher noise, and so get
74 flattened out when normalized.
75
76 |compare_residual_normalize_png|
77
78 #. Right click on the plot and use the Axes menu to change the scale of the axes.
79 Try changing from a Log-Lin to a Log-Log plot to emphasize the low q.
80
81 * *Note:* You can change the plot scale in the Ratios plot as well.
82
83 #. Close the comparison window using the Close button.
84
85 Comparing profiles with ratios
86 #################################
87
88 The ratio of two profiles can be useful when looking for small changes between
89 samples. It's often used for looking for concentration dependent effects,
90 particularly inter-particle interactions which cause a characteristic downturn
91 at low q.
92
93 The written version of this tutorial follows.
94
95 #. Clear any data loaded into RAW. Load the **S_Lys05mg.dat**, **S_Lys28mg.dat**,
96 and **S_Lys47mg.dat** profiles into RAW. These are scattering profiles of
97 lysozyme measured at three different concentrations, 5 mg/ml, 28 mg/ml, and
98 47 mg/ml.
99
100 * *Note:* The profiles should have significantly different overall intensities,
101 due to the different concentrations.
102
103 * *Note:* In order to remove some high q noise, profiles have been truncated
104 to q=0.2.
105
106 #. Select all three profiles, right click and select "Compare Profiles" to open
107 the comparison window.
108
109 #. Select the "Ratios" tab at the top of the window.
110
111 * *Note:* The ratios shown are ratios to the reference profile. The choice
112 of reference can be changed as in the Residuals tab.
113
114 |compare_ratios_tab_png|
115
116 #. Because of the different intensity levels that the profiles have due to the
117 different concentrations the ratios aren't as useful for examining what's
118 going on at low q. RAW can automatically apply a scale factor to the profiles.
119 In the "Scale" drop-down menu select "Scale, high q", which will scale the
120 profiles to the reference in the high q region of the data.
121
122 * *Note:* There are a number of scale options available, which can be
123 useful in different situations. The Residuals tab also has the scale
124 option.
125
126 * *Note:* You can see that there is a significant downturn at low q in both
127 the 28 mg/ml and 47 mg/ml profile, indicating the presence of significant
128 repulsion in the sample.
129
130 |compare_ratios_scale_png|
131
132 #. If you want to examine just one of the ratios, you can turn off a profile.
133 Uncheck the **S_Lys47mg.dat** profile to show just the ratio of the 28 mg/ml to
134 5 mg/ml data.
135
136 |compare_ratios_show_png|
137
138 #. Close the comparison window using the Close button.
139
140 Comparing profiles with statistical tests
141 ############################################
142
143 In addition to the visual comparisons of the residual and ratio, above, RAW
144 has the ability to test scattering profiles for statistical similarity. Currently, only one
145 test is available: the Correlation Map test. This can be done manually, and is also done
146 automatically when scattering profiles are averaged. This can be useful when you’re dealing
147 with data that may show changes in scattering from radiation damage or other possible sources.
148
149 A video version of an older version this tutorial is available:
150
151 .. raw:: html
152
153 <style>.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }</style><div class='embed-container'><iframe src='https://www.youtube.com/embed/BPgWze8GjVI' frameborder='0' allowfullscreen></iframe></div>
154
155 The written version of the tutorial follows.
156
157 #. Clear any data loaded into RAW. Load all of the profiles in the **Tutorial_Data/damage_data**
158 folder into the Profiles plot. Show only the top plot.
159
160 * *Tip:* In the Files tab, click the “Clear All” button.
161
162 #. Put the plot on a log-log scale. You should see that the profiles are different at low *q*\ .
163
164 * *Note:* These data are showing what radiation damage looks like in a data set. They
165 are consecutive profiles from the same sample, and as total exposure of the sample
166 increase (frame number increases), the sample damages. In this case, the damage
167 is manifesting as aggregation, which shows up as an uptick in the profiles at low *q*\ .
168
169 |similarity_main_png|
170
171 #. Select all of the profiles and average them. You will get a warning message informing you
172 that not all the files are statistically the same.
173
174 * *Note:* This is only as good as the statistical test being used, and the cutoff
175 threshold selected. In the advanced options panel you can select the test, whether
176 or not it is corrected for multiple testing, and the threshold used.
177
178 |similarity_warning_png|
179
180 #. Click the “Average Only Similar Files” button.
181
182 * Note: This averages only those profiles found to the same as the first file,
183 for the given statistical test.
184
185 #. Select all of the profiles except the new averaged one, and right click and
186 select “Compare Profiles”. Select the "Similarity Test" tab at the top of the
187 window. This tab shows the results of the pairwise tests done using the
188 CorMap method. On the top is a heatmap of the probability that the profiles
189 are similar, while on the bottom is a list of the pairwise comparisons
190 and the actual probabilities.
191
192 |similarity_window_png|
193
194 #. Expand the Filename columns in the list so you can see the full filenames along
195 with the probabilities.
196
197 |similarity_window2_png|
198
199 #. Using the menu at the top, turn off multiple testing correction. Change the
200 "highlight with p-value <" value to 0.15, and highlight those pairs.
201
202 * *Note:* Multiple testing correction corrects for the fact that you expect
203 more outliers the more pairwise tests you run. Suppose the distribution
204 of the uncertainty of a variable is Gaussian. If you run a single test
205 to compare the results between two measurements of the variable,
206 getting a result that is 2 standard deviations from the mean is a
207 significant difference. However, if you make 100 measurements and
208 compare them all to the first, you expect that 5% of the measurements
209 you make will be outside of 2 standard deviations different, so a multiple
210 testing correction attempts to correct for that factor.
211
212 |similarity_highlight_png|
213
214 #. Move you mouse across the heatmap. Note that the profile numbers and probability
215 value is displayed in the plot toolbar below the plot. The "X" refers to the
216 File # of the file being compared on the x axis, the "Y" refers to the File #
217 of the file being compared on the y axis, the "Prob." is the probability
218 (corrected, if multiple testing correction is turned on) that the two are
219 the same, and the "Test val" is the actual value of the comparison test,
220 in this case the CorMap longest edge.
221
222 * *Note:* Comparing file x against file y is the same as comparing file
223 y against file x, so the heatmap is symmetric across the diagonal. The
224 diagonal values are comparing a profile to itself, and so the probability
225 is always 1 along the diagonal.
226
227 |similarity_heatmap_png|
228
229 #. Without multiple testing correction, and using a less stringent threshold for similarity,
230 we see that more profiles are selected here (profiles 6-10) than were excluded from the
231 average using the automatic test. Because we know radiation damage increases with dose,
232 it is reasonable to suspect that we should discard profiles 6-10, not just 8-10 as in
233 the automated version.
234
235 #. Save the similarity test data in the list as a **.csv** by clicking the “Save” button.
236
237 #. Close the comparison window by clicking the “Done” button.
238
239 #. Average profiles 1-5.
240
241 #. Hide all of the profiles except the two averaged profiles on the plot.
242
243 * *Question:* Is there a difference between the two? What about if you do a Guinier fit?
244
245 * *Note:* In this case, the differences are subtle, a ~1-2% increase in |Rg|. So
246 the automated determination did a reasonable job. However, it is generally good
247 to double check your set of profiles both visually and using the Similarity Test
248 panel when the automated test warns you of outlier profiles.
249
250
251
252
253 .. |compare_residual_right_click_png| image:: images/compare_residual_right_click.png
254 :target: ../_images/compare_residual_right_click.png
255 :width: 300 px
256
257 .. |compare_residual_main_png| image:: images/compare_residual_main.png
258 :target: ../_images/compare_residual_main.png
259
260 .. |compare_residual_ref_png| image:: images/compare_residual_ref.png
261 :target: ../_images/compare_residual_ref.png
262
263 .. |compare_residual_normalize_png| image:: images/compare_residual_normalize.png
264 :target: ../_images/compare_residual_normalize.png
265 :width: 300 px
266
267 .. |compare_ratios_tab_png| image:: images/compare_ratios_tab.png
268 :target: ../_images/compare_ratios_tab.png
269
270 .. |compare_ratios_scale_png| image:: images/compare_ratios_scale.png
271 :target: ../_images/compare_ratios_scale.png
272
273 .. |compare_ratios_show_png| image:: images/compare_ratios_show.png
274 :target: ../_images/compare_ratios_show.png
275 :width: 300 px
276
277 .. |similarity_main_png| image:: images/similarity_main.png
278 :target: ../_images/similarity_main.png
279
280 .. |similarity_warning_png| image:: images/similarity_warning.png
281 :width: 500 px
282 :target: ../_images/similarity_warning.png
283
284 .. |similarity_window_png| image:: images/similarity_window.png
285 :width: 600 px
286 :target: ../_images/similarity_window.png
287
288 .. |similarity_window2_png| image:: images/similarity_window2.png
289 :target: ../_images/similarity_window2.png
290
291 .. |similarity_highlight_png| image:: images/similarity_highlight.png
292 :target: ../_images/similarity_highlight.png
293
294 .. |similarity_heatmap_png| image:: images/similarity_heatmap.png
295 :target: ../_images/similarity_heatmap.png
296 :width: 400 px
297
298 .. |Rg| replace:: R\ :sub:`g`