| 1 |
Comparing profiles |
| 2 |
^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 3 |
.. _s1p6: |
| 4 |
|
| 5 |
It is often useful to compare how similar profiles are, visually and/or statistically. |
| 6 |
RAW has a dedicated comparison window that allows you to compare residuals, between |
| 7 |
profiles, ratios between profiles, and use statistical tests (currently only the |
| 8 |
Correlation Map test is implemented) to check for differences between profiles. |
| 9 |
|
| 10 |
|
| 11 |
Comparing profiles with residuals |
| 12 |
#################################### |
| 13 |
|
| 14 |
The residual is the difference between two profiles. Often we normalize this |
| 15 |
difference by the uncertainty of one of the profiles, so that large deviations |
| 16 |
of points with large uncertainties don't dominate the residual. If two profiles |
| 17 |
agree within the noise level of the data the residual should be flat and |
| 18 |
randomly distributed. If normalized, you expect most of the residual values to |
| 19 |
be within +/- 2.5. It is often used when comparing a fit against the data |
| 20 |
(e.g. comparing the fit of the P(r) transformed to I(q) or of a reconstruction |
| 21 |
against the data). |
| 22 |
|
| 23 |
The written version of the tutorial follows. |
| 24 |
|
| 25 |
#. Clear any data loaded into RAW. Load the **2pol_polymerase.fit** data |
| 26 |
in the **Tutorial_Data/theory_data/theory_results** folder into the |
| 27 |
Profiles plot. |
| 28 |
|
| 29 |
* *Note:* This fit will load in two profiles, experimental data and the |
| 30 |
CRYSOL fit of a PDB file (high resolution model) to that experimental |
| 31 |
data. See the :ref:`CRYSOL tutorial <crysol_tutorial>` for how to generate |
| 32 |
these fits. |
| 33 |
|
| 34 |
#. Select both the **2pol_polymerase.fit** (experimental data) and |
| 35 |
**2pol_polymerase_FIT** (theoretical model fit to the data) profiles, |
| 36 |
right click on one of them and select "Compare Profiles" in the right |
| 37 |
click menu. |
| 38 |
|
| 39 |
|compare_residual_right_click_png| |
| 40 |
|
| 41 |
#. The Compare Profiles window will open. You will see three tabs at the top, |
| 42 |
the first (and default) shows the residuals between the selected data. |
| 43 |
|
| 44 |
* *Note:* You can select as many profiles as you want for comparison, |
| 45 |
not just two. |
| 46 |
|
| 47 |
* *Tip:* You can use the checkboxes next to the profile names to turn on |
| 48 |
profiles in the comparison plot. You must have at least two profiles |
| 49 |
turned on or no residuals will be displayed. |
| 50 |
|
| 51 |
|compare_residual_main_png| |
| 52 |
|
| 53 |
#. Use the "Ref. profile" drop down menu to select the **2pol_polymerase_FIT** |
| 54 |
profile as the reference profile. The reference profile is the profile that |
| 55 |
is subtracted from every other profile. Here you use the theoretical data |
| 56 |
as the reference. |
| 57 |
|
| 58 |
* *Note:* The profile plot will update so that the reference profile is |
| 59 |
bold and on top of all the other plotted profiles. |
| 60 |
|
| 61 |
* *Note:* You can see that the residuals have a trend to them, they're |
| 62 |
above zero at high q (0.15-0.2) and then dip below at mid q (around 0.13). |
| 63 |
This indicates that the theoretical model, while mostly a good fit, isn't |
| 64 |
a perfect fit for the experimental data. |
| 65 |
|
| 66 |
|compare_residual_ref_png| |
| 67 |
|
| 68 |
#. Use the "Normalize Residuals" checkbox to turn normalization on and off and |
| 69 |
see what difference it makes for both the overall values and shape of the |
| 70 |
residual. |
| 71 |
|
| 72 |
* *Note:* You should see the un-normalized residual becomes more dominated |
| 73 |
by very low q deviations, but those points are higher noise, and so get |
| 74 |
flattened out when normalized. |
| 75 |
|
| 76 |
|compare_residual_normalize_png| |
| 77 |
|
| 78 |
#. Right click on the plot and use the Axes menu to change the scale of the axes. |
| 79 |
Try changing from a Log-Lin to a Log-Log plot to emphasize the low q. |
| 80 |
|
| 81 |
* *Note:* You can change the plot scale in the Ratios plot as well. |
| 82 |
|
| 83 |
#. Close the comparison window using the Close button. |
| 84 |
|
| 85 |
Comparing profiles with ratios |
| 86 |
################################# |
| 87 |
|
| 88 |
The ratio of two profiles can be useful when looking for small changes between |
| 89 |
samples. It's often used for looking for concentration dependent effects, |
| 90 |
particularly inter-particle interactions which cause a characteristic downturn |
| 91 |
at low q. |
| 92 |
|
| 93 |
The written version of this tutorial follows. |
| 94 |
|
| 95 |
#. Clear any data loaded into RAW. Load the **S_Lys05mg.dat**, **S_Lys28mg.dat**, |
| 96 |
and **S_Lys47mg.dat** profiles into RAW. These are scattering profiles of |
| 97 |
lysozyme measured at three different concentrations, 5 mg/ml, 28 mg/ml, and |
| 98 |
47 mg/ml. |
| 99 |
|
| 100 |
* *Note:* The profiles should have significantly different overall intensities, |
| 101 |
due to the different concentrations. |
| 102 |
|
| 103 |
* *Note:* In order to remove some high q noise, profiles have been truncated |
| 104 |
to q=0.2. |
| 105 |
|
| 106 |
#. Select all three profiles, right click and select "Compare Profiles" to open |
| 107 |
the comparison window. |
| 108 |
|
| 109 |
#. Select the "Ratios" tab at the top of the window. |
| 110 |
|
| 111 |
* *Note:* The ratios shown are ratios to the reference profile. The choice |
| 112 |
of reference can be changed as in the Residuals tab. |
| 113 |
|
| 114 |
|compare_ratios_tab_png| |
| 115 |
|
| 116 |
#. Because of the different intensity levels that the profiles have due to the |
| 117 |
different concentrations the ratios aren't as useful for examining what's |
| 118 |
going on at low q. RAW can automatically apply a scale factor to the profiles. |
| 119 |
In the "Scale" drop-down menu select "Scale, high q", which will scale the |
| 120 |
profiles to the reference in the high q region of the data. |
| 121 |
|
| 122 |
* *Note:* There are a number of scale options available, which can be |
| 123 |
useful in different situations. The Residuals tab also has the scale |
| 124 |
option. |
| 125 |
|
| 126 |
* *Note:* You can see that there is a significant downturn at low q in both |
| 127 |
the 28 mg/ml and 47 mg/ml profile, indicating the presence of significant |
| 128 |
repulsion in the sample. |
| 129 |
|
| 130 |
|compare_ratios_scale_png| |
| 131 |
|
| 132 |
#. If you want to examine just one of the ratios, you can turn off a profile. |
| 133 |
Uncheck the **S_Lys47mg.dat** profile to show just the ratio of the 28 mg/ml to |
| 134 |
5 mg/ml data. |
| 135 |
|
| 136 |
|compare_ratios_show_png| |
| 137 |
|
| 138 |
#. Close the comparison window using the Close button. |
| 139 |
|
| 140 |
Comparing profiles with statistical tests |
| 141 |
############################################ |
| 142 |
|
| 143 |
In addition to the visual comparisons of the residual and ratio, above, RAW |
| 144 |
has the ability to test scattering profiles for statistical similarity. Currently, only one |
| 145 |
test is available: the Correlation Map test. This can be done manually, and is also done |
| 146 |
automatically when scattering profiles are averaged. This can be useful when you’re dealing |
| 147 |
with data that may show changes in scattering from radiation damage or other possible sources. |
| 148 |
|
| 149 |
A video version of an older version this tutorial is available: |
| 150 |
|
| 151 |
.. raw:: html |
| 152 |
|
| 153 |
<style>.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }</style><div class='embed-container'><iframe src='https://www.youtube.com/embed/BPgWze8GjVI' frameborder='0' allowfullscreen></iframe></div> |
| 154 |
|
| 155 |
The written version of the tutorial follows. |
| 156 |
|
| 157 |
#. Clear any data loaded into RAW. Load all of the profiles in the **Tutorial_Data/damage_data** |
| 158 |
folder into the Profiles plot. Show only the top plot. |
| 159 |
|
| 160 |
* *Tip:* In the Files tab, click the “Clear All” button. |
| 161 |
|
| 162 |
#. Put the plot on a log-log scale. You should see that the profiles are different at low *q*\ . |
| 163 |
|
| 164 |
* *Note:* These data are showing what radiation damage looks like in a data set. They |
| 165 |
are consecutive profiles from the same sample, and as total exposure of the sample |
| 166 |
increase (frame number increases), the sample damages. In this case, the damage |
| 167 |
is manifesting as aggregation, which shows up as an uptick in the profiles at low *q*\ . |
| 168 |
|
| 169 |
|similarity_main_png| |
| 170 |
|
| 171 |
#. Select all of the profiles and average them. You will get a warning message informing you |
| 172 |
that not all the files are statistically the same. |
| 173 |
|
| 174 |
* *Note:* This is only as good as the statistical test being used, and the cutoff |
| 175 |
threshold selected. In the advanced options panel you can select the test, whether |
| 176 |
or not it is corrected for multiple testing, and the threshold used. |
| 177 |
|
| 178 |
|similarity_warning_png| |
| 179 |
|
| 180 |
#. Click the “Average Only Similar Files” button. |
| 181 |
|
| 182 |
* Note: This averages only those profiles found to the same as the first file, |
| 183 |
for the given statistical test. |
| 184 |
|
| 185 |
#. Select all of the profiles except the new averaged one, and right click and |
| 186 |
select “Compare Profiles”. Select the "Similarity Test" tab at the top of the |
| 187 |
window. This tab shows the results of the pairwise tests done using the |
| 188 |
CorMap method. On the top is a heatmap of the probability that the profiles |
| 189 |
are similar, while on the bottom is a list of the pairwise comparisons |
| 190 |
and the actual probabilities. |
| 191 |
|
| 192 |
|similarity_window_png| |
| 193 |
|
| 194 |
#. Expand the Filename columns in the list so you can see the full filenames along |
| 195 |
with the probabilities. |
| 196 |
|
| 197 |
|similarity_window2_png| |
| 198 |
|
| 199 |
#. Using the menu at the top, turn off multiple testing correction. Change the |
| 200 |
"highlight with p-value <" value to 0.15, and highlight those pairs. |
| 201 |
|
| 202 |
* *Note:* Multiple testing correction corrects for the fact that you expect |
| 203 |
more outliers the more pairwise tests you run. Suppose the distribution |
| 204 |
of the uncertainty of a variable is Gaussian. If you run a single test |
| 205 |
to compare the results between two measurements of the variable, |
| 206 |
getting a result that is 2 standard deviations from the mean is a |
| 207 |
significant difference. However, if you make 100 measurements and |
| 208 |
compare them all to the first, you expect that 5% of the measurements |
| 209 |
you make will be outside of 2 standard deviations different, so a multiple |
| 210 |
testing correction attempts to correct for that factor. |
| 211 |
|
| 212 |
|similarity_highlight_png| |
| 213 |
|
| 214 |
#. Move you mouse across the heatmap. Note that the profile numbers and probability |
| 215 |
value is displayed in the plot toolbar below the plot. The "X" refers to the |
| 216 |
File # of the file being compared on the x axis, the "Y" refers to the File # |
| 217 |
of the file being compared on the y axis, the "Prob." is the probability |
| 218 |
(corrected, if multiple testing correction is turned on) that the two are |
| 219 |
the same, and the "Test val" is the actual value of the comparison test, |
| 220 |
in this case the CorMap longest edge. |
| 221 |
|
| 222 |
* *Note:* Comparing file x against file y is the same as comparing file |
| 223 |
y against file x, so the heatmap is symmetric across the diagonal. The |
| 224 |
diagonal values are comparing a profile to itself, and so the probability |
| 225 |
is always 1 along the diagonal. |
| 226 |
|
| 227 |
|similarity_heatmap_png| |
| 228 |
|
| 229 |
#. Without multiple testing correction, and using a less stringent threshold for similarity, |
| 230 |
we see that more profiles are selected here (profiles 6-10) than were excluded from the |
| 231 |
average using the automatic test. Because we know radiation damage increases with dose, |
| 232 |
it is reasonable to suspect that we should discard profiles 6-10, not just 8-10 as in |
| 233 |
the automated version. |
| 234 |
|
| 235 |
#. Save the similarity test data in the list as a **.csv** by clicking the “Save” button. |
| 236 |
|
| 237 |
#. Close the comparison window by clicking the “Done” button. |
| 238 |
|
| 239 |
#. Average profiles 1-5. |
| 240 |
|
| 241 |
#. Hide all of the profiles except the two averaged profiles on the plot. |
| 242 |
|
| 243 |
* *Question:* Is there a difference between the two? What about if you do a Guinier fit? |
| 244 |
|
| 245 |
* *Note:* In this case, the differences are subtle, a ~1-2% increase in |Rg|. So |
| 246 |
the automated determination did a reasonable job. However, it is generally good |
| 247 |
to double check your set of profiles both visually and using the Similarity Test |
| 248 |
panel when the automated test warns you of outlier profiles. |
| 249 |
|
| 250 |
|
| 251 |
|
| 252 |
|
| 253 |
.. |compare_residual_right_click_png| image:: images/compare_residual_right_click.png |
| 254 |
:target: ../_images/compare_residual_right_click.png |
| 255 |
:width: 300 px |
| 256 |
|
| 257 |
.. |compare_residual_main_png| image:: images/compare_residual_main.png |
| 258 |
:target: ../_images/compare_residual_main.png |
| 259 |
|
| 260 |
.. |compare_residual_ref_png| image:: images/compare_residual_ref.png |
| 261 |
:target: ../_images/compare_residual_ref.png |
| 262 |
|
| 263 |
.. |compare_residual_normalize_png| image:: images/compare_residual_normalize.png |
| 264 |
:target: ../_images/compare_residual_normalize.png |
| 265 |
:width: 300 px |
| 266 |
|
| 267 |
.. |compare_ratios_tab_png| image:: images/compare_ratios_tab.png |
| 268 |
:target: ../_images/compare_ratios_tab.png |
| 269 |
|
| 270 |
.. |compare_ratios_scale_png| image:: images/compare_ratios_scale.png |
| 271 |
:target: ../_images/compare_ratios_scale.png |
| 272 |
|
| 273 |
.. |compare_ratios_show_png| image:: images/compare_ratios_show.png |
| 274 |
:target: ../_images/compare_ratios_show.png |
| 275 |
:width: 300 px |
| 276 |
|
| 277 |
.. |similarity_main_png| image:: images/similarity_main.png |
| 278 |
:target: ../_images/similarity_main.png |
| 279 |
|
| 280 |
.. |similarity_warning_png| image:: images/similarity_warning.png |
| 281 |
:width: 500 px |
| 282 |
:target: ../_images/similarity_warning.png |
| 283 |
|
| 284 |
.. |similarity_window_png| image:: images/similarity_window.png |
| 285 |
:width: 600 px |
| 286 |
:target: ../_images/similarity_window.png |
| 287 |
|
| 288 |
.. |similarity_window2_png| image:: images/similarity_window2.png |
| 289 |
:target: ../_images/similarity_window2.png |
| 290 |
|
| 291 |
.. |similarity_highlight_png| image:: images/similarity_highlight.png |
| 292 |
:target: ../_images/similarity_highlight.png |
| 293 |
|
| 294 |
.. |similarity_heatmap_png| image:: images/similarity_heatmap.png |
| 295 |
:target: ../_images/similarity_heatmap.png |
| 296 |
:width: 400 px |
| 297 |
|
| 298 |
.. |Rg| replace:: R\ :sub:`g` |