| 1 |
Advanced Series processing – Evolving factor analysis (EFA) |
| 2 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 3 |
.. _raw_efa: |
| 4 |
|
| 5 |
Sometimes SEC fails to fully separate out different species, and you end up with overlapping |
| 6 |
peaks in your SEC-SAXS curve. It is possible to apply more advanced mathematical techniques |
| 7 |
to determine if there are multiple species of macromolecule in a SEC-SAXS peak, and to attempt |
| 8 |
to extract out scattering profiles for each component in an overlapping peak. |
| 9 |
:ref:`Singular value decomposition (SVD) <raw_svd>` can be used to help determine how many distinct scatterers are in a |
| 10 |
SEC-SAXS peak. Evolving factor analysis (EFA) is an extension of SVD that can extract individual |
| 11 |
components from overlapping SEC-SAXS peaks. Note that the first step of EFA is |
| 12 |
doing SVD, but that happens entirely within the EFA analysis window. The SVD |
| 13 |
window does not need to be opened before doing EFA. This tutorial covers |
| 14 |
EFA. |
| 15 |
|
| 16 |
:ref:`REGALS <raw_regals>` is a similar deconvolution technique, but can be |
| 17 |
applied in cases where there are components that are not strictly |
| 18 |
first-in-first-out and EFA would fail. EFA is recommended for standard |
| 19 |
SEC-SAXS data, but for more complex data, such as ion exchange chromatography, |
| 20 |
or time resolved or titration data you should use REGALS. REGALS can also |
| 21 |
handle deconvolution of SEC-SAXS data with a sloping baseline. |
| 22 |
|
| 23 |
If you use EFA in RAW, in addition to citing the RAW paper, please cite the |
| 24 |
EFA paper: S. P. Meisburger, A. B. Taylor, C. A. Khan, S. Zhang, P. F. |
| 25 |
Fitzpatrick, N. Ando. Journal of the American Chemical Society (2016). 138(20), |
| 26 |
6506-6516. DOI: `10.1021/jacs.6b01563 <https://doi.org/10.1021/jacs.6b01563>`_ |
| 27 |
|
| 28 |
A video version of this tutorial is available: |
| 29 |
|
| 30 |
.. raw:: html |
| 31 |
|
| 32 |
<style>.embed-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; } .embed-container iframe, .embed-container object, .embed-container embed { position: absolute; top: 0; left: 0; width: 100%; height: 100%; }</style><div class='embed-container'><iframe src='https://www.youtube.com/embed/U2bSg20mU8s' frameborder='0' allowfullscreen></iframe></div> |
| 33 |
|
| 34 |
The written version of the tutorial follows. |
| 35 |
|
| 36 |
|
| 37 |
#. Clear all of the data in RAW. Load the **phehc_sec.hdf5** file in the **series_data** folder. |
| 38 |
|
| 39 |
* *Note:* The data were provided by the Ando group at Cornell University |
| 40 |
and is some of the data used in the paper: *Domain Movements upon Activation of |
| 41 |
Phenylalanine Hydroxylase Characterized by Crystallography and Chromatography-Coupled |
| 42 |
Small-Angle X-ray Scattering*\ . Steve P. Meisburger, Alexander B. Taylor, Crystal |
| 43 |
A. Khan, Shengnan Zhang, Paul F. Fitzpatrick, and Nozomi Ando. Journal of the |
| 44 |
American Chemical Society 2016 138 (20), 6506-6516. `DOI: 10.1021/jacs.6b01563 |
| 45 |
<https://dx.doi.org/10.1021/jacs.6b01563>`_ |
| 46 |
|
| 47 |
|efa_series_plot_png| |
| 48 |
|
| 49 |
#. We will use EFA to extract out the two scattering components in the main |
| 50 |
peak in this data. Right click on the **phehc_sec.hdf5** item in the Series |
| 51 |
list. Select the “EFA” option. |
| 52 |
|
| 53 |
#. The EFA window will be displayed. On the left are controls, on the right are plots of |
| 54 |
the value of the singular values and the first autocorrelation of the left and right |
| 55 |
singular vectors. |
| 56 |
|
| 57 |
* *Note:* Large singular values indicate significant components. What matters is the relative |
| 58 |
magnitude, that is, whether the value is large relative to the mostly flat/unchanging |
| 59 |
value of high index singular values. |
| 60 |
|
| 61 |
* *Note:* A large autocorrelation indicates that the singular vector is varying smoothly, |
| 62 |
while a low autocorrelation indicates the vector is very noisy. Vectors corresponding to |
| 63 |
significant components will tend to have autocorrelations near 1 (roughly, >0.6-0.7) and |
| 64 |
vectors corresponding to insignificant components will tend to have autocorrelations near 0. |
| 65 |
|
| 66 |
|efa_panel_png| |
| 67 |
|
| 68 |
#. For successful EFA, you want to use Subtracted data, and you often want to have |
| 69 |
a buffer region before and after the sample. For this data set, using the entire |
| 70 |
frame range (from 0 to 385) is appropriate. With other data sets, you may need to |
| 71 |
change the frame range to, for example, remove other, well separated, peaks from the |
| 72 |
analysis. |
| 73 |
|
| 74 |
* *Tip:* If you have a dataset where you have a large number of components, |
| 75 |
such as 4+, it can be useful to set the EFA range to isolate just |
| 76 |
2-3 of those components. The more components you have, the harder |
| 77 |
it is to do the EFA. There is a trade off in the amount of data |
| 78 |
used (more is better), and the number of components in the |
| 79 |
deconvolution (less is better) that requires some experimentation |
| 80 |
to find the right balance for a given dataset. |
| 81 |
|
| 82 |
#. RAW attempts to automatically determine how many significant singular values (SVs) there |
| 83 |
are in the selected range. This corresponds to the number of significant scattering |
| 84 |
components in solution that EFA will attempt to deconvolve. At the bottom of |
| 85 |
the control panel, you should see that RAW thinks there are three significant |
| 86 |
SVs (scattering components) in our data. For this data set, that is accurate. |
| 87 |
We evaluate the number of significant components by how many singular values |
| 88 |
are above the baseline, and how many components have both left and right singular |
| 89 |
vectors with autocorrelations near one. For this data there are three singular |
| 90 |
values above baseline, and three singular vectors with autocorrelations near |
| 91 |
1 (see step 3). |
| 92 |
|
| 93 |
|efa_components_png| |
| 94 |
|
| 95 |
* *Note:* Typically you want the number of significant singular values and |
| 96 |
the number of singular vectors with autocorrelations near 1 to be equal. |
| 97 |
If they aren't, it likely indicates a weak or otherwise poorly resolved |
| 98 |
component in the dataset. Try the deconvolution first with the lower then |
| 99 |
the higher number of components. |
| 100 |
|
| 101 |
* *Note:* RAW can find the wrong number of components automatically. You will |
| 102 |
always want to double check this automatic determination against the SVD results in |
| 103 |
the plots. If you change the data range used (or data type), the number |
| 104 |
of components will not automatically update so you should check and update |
| 105 |
it if necessary. |
| 106 |
|
| 107 |
#. Click the “Next” button in the lower right-hand corner of the window to advance to |
| 108 |
the second stage of the EFA analysis. |
| 109 |
|
| 110 |
* *Note:* It may take some time to compute the necessary values for this next step, |
| 111 |
so be patient. |
| 112 |
|
| 113 |
|efa_panel_2_png| |
| 114 |
|
| 115 |
#. This step shows you the “Forward EFA” and “Backward EFA” plots. These plots represent |
| 116 |
the value of the singular values as a function of frame. |
| 117 |
|
| 118 |
* *Note:* There is one more singular value displayed on each plot than available in |
| 119 |
the controls. This is so that in the following Steps you can determine where each |
| 120 |
component deviates from the baseline. |
| 121 |
|
| 122 |
#. In the User Input panel, tweak the “Forward” value start frames so that the frame |
| 123 |
number, as indicated by the open circle on the plot, aligns with where the singular |
| 124 |
value first starts to increase quickly. This should be around 147, 164, and 322. |
| 125 |
|
| 126 |
* *Note:* For the Forward EFA plot, SVD is run on just the first two frames, then |
| 127 |
the first three, and so on, until all frames in the range are included. As more |
| 128 |
frames are added, the singular values change, as shown on the plot. When a singular |
| 129 |
value starts increasingly sharply, it indicates that there is a new scattering |
| 130 |
component in the scattering profile measured at that point. So, for the first ~150 |
| 131 |
frames, there are no new scattering components (i.e. just buffer scattering). At |
| 132 |
frame ~147, we see the first singular value (the singular value with index 0, |
| 133 |
labeled SV 0 on the plot) start to strongly increase, showing that we have gained |
| 134 |
a scattering component. We see SV 1 start to increase at ~164, indicating another |
| 135 |
scattering component starting to be present in the data. |
| 136 |
|
| 137 |
#. In the User Input panel, tweak the “Backward” value start frames so that the frame |
| 138 |
number, as indicated by the open circle on the plot, aligns with where |
| 139 |
the singular value drops back to near the baseline. This should be around |
| 140 |
383, 360, and 200. |
| 141 |
|
| 142 |
* *Note:* For the Backward EFA plot, SVD is run on just the last two frames, then the |
| 143 |
last three, and so on, until all frames in the range are included. As more frames are |
| 144 |
added, the singular values change, as shown on the plot. When a singular value |
| 145 |
drops back to baseline, it indicates that a scattering component is leaving |
| 146 |
the dataset at that point. |
| 147 |
|
| 148 |
* *Note:* The algorithm for determining the start and end points is not particularly |
| 149 |
advanced. For some datasets you may need to do significantly more adjustment of these values |
| 150 |
|
| 151 |
|efa_ranges_png| |
| 152 |
|
| 153 |
#. Click the “Next” button in the bottom right corner to move to the last stage of the |
| 154 |
EFA analysis. |
| 155 |
|
| 156 |
|efa_panel_3_png| |
| 157 |
|
| 158 |
#. This window shows controls on the left and results on the right. In the controls area, |
| 159 |
at the top is a plot showing the SEC-SAXS curve, along with the ranges occupied by |
| 160 |
each scattering component, as determined from the input on the Forward and Backward |
| 161 |
EFA curves in stage 2 of the analysis. The colors of the ranges correspond to the |
| 162 |
colors labeled in the Scattering Profiles plot on the top right and the Concentration |
| 163 |
plot in the lower right. This panel takes the SVD vectors and rotates them back into |
| 164 |
scattering vectors corresponding to real components. |
| 165 |
|
| 166 |
* *Note:* This rotation is not guaranteed to be successful, or to give you valid |
| 167 |
scattering vectors. Any data obtained via this method should be supported in other |
| 168 |
ways, either using other methods of deconvolving the peak, other biophysical or |
| 169 |
biochemical data, or both! |
| 170 |
|
| 171 |
#. This rotation looks quite good, as judged by the reasonable profiles, concentration |
| 172 |
peaks, and relatively flat chi^2 vs. frame plot. However, you don't always pick the |
| 173 |
right ranges the first time, sometimes some fine tuning is necessary. To simulate |
| 174 |
this, in the “Component Range Controls” set the ranges back to the original |
| 175 |
default values found by RAW: 151 to 193, 164 to 322, and 319 to 347. |
| 176 |
|
| 177 |
#. After making these adjustments, you should see some spikes in the chi^2 values. |
| 178 |
|
| 179 |
|efa_poor_rotation_png| |
| 180 |
|
| 181 |
#. Fine tune the ranges using the controls in the “Component Range Controls” box. |
| 182 |
Adjust the starts and ends of Ranges 0 and 1 and the end of Range 2 by a few points |
| 183 |
until the spikes in the chi-squared plot go away. After these adjustments, Range 0 |
| 184 |
should be about 142 to 198, Range 1 from 161 to 322, and Range 2 from 319 to 360. |
| 185 |
|
| 186 |
* *Note:* These ranges are a little different from what you previously found, |
| 187 |
particularly the end of ranges 1 and 2. This likely means that there is |
| 188 |
very little (or no) contribution of those components in the extended range. |
| 189 |
You can verify this by setting the ends of those ranges back to 360 and 383 |
| 190 |
respectively and looking at the concentration profiles, you'll see that the |
| 191 |
profiles are essentially zero in the more extended ranges. It's usually |
| 192 |
a good idea to minimize the component range to avoid introducing contamination |
| 193 |
from other components. In this case, you could do that by narrowing the |
| 194 |
range of the components until you start to see chi^2 spikes, then returning |
| 195 |
to the last good value. |
| 196 |
|
| 197 |
#. To see these changes on the Forward and Backward EFA plots, click the “Back” button |
| 198 |
at the bottom right of the page. Verify that all of your start and end values are |
| 199 |
close to where the components become significant, as discussed in Steps 8 and 9. |
| 200 |
|
| 201 |
#. Click the “Next” button to return to the final stage of the EFA analysis. |
| 202 |
|
| 203 |
#. In the Rotation Controls box, you can set the method, the number of iterations, the |
| 204 |
convergence threshold, and whether you're starting with the previous results. |
| 205 |
As you can see in the Status window, the rotation was successful for this |
| 206 |
data. If it was not, you could try changing methods or adjusting the number |
| 207 |
of iterations or threshold. |
| 208 |
|
| 209 |
* *Tip:* If it takes a while to run EFA every time you change a component, |
| 210 |
you can speed up the convergence by starting with the previous results. |
| 211 |
To do so, you would check the "Start with previous results" box. This |
| 212 |
will allow you to quickly iterate on changes, as long as the magnitude |
| 213 |
of the change is relatively small. Just be sure to set the convergence |
| 214 |
criteria back to not using previous results to do your final EFA run, |
| 215 |
as you can bias the rotation with the previous results and guide the |
| 216 |
EFA into a solution that is path dependent and thus isn't reproducible later. |
| 217 |
|
| 218 |
#. Examine the chi-squared plot. It should be uniformly close to 1 for good EFA. For |
| 219 |
this data, it is. |
| 220 |
|
| 221 |
#. Examine the concentration plot. You’ll see three peaks, corresponding to the |
| 222 |
concentrations for the three components. In the Range Controls, uncheck the Range |
| 223 |
0 C>=0 box. That removes the constraint that the concentration must be positive. |
| 224 |
If this results in a significant change in the peak, your EFA analysis is likely |
| 225 |
poor, and you should not trust your results. |
| 226 |
|
| 227 |
* *Note:* The height of the concentration peaks is arbitrary, all peaks are |
| 228 |
normalized to have an area of 1. |
| 229 |
|
| 230 |
#. Uncheck all of the C>=0 controls. |
| 231 |
|
| 232 |
* *Question:* Do you observe any significant changes in the scattering profiles, |
| 233 |
chi-squared, or concentration when you do this? How about if you uncheck one and |
| 234 |
leave the others checked? |
| 235 |
|
| 236 |
#. Recheck all of the C>=0 controls. You have now verified, as much as you can, that |
| 237 |
the EFA analysis is giving you reasonable results. |
| 238 |
|
| 239 |
#. *Reminder:* Here are the verification steps we have carried out, and you should carry |
| 240 |
out every time you do EFA: |
| 241 |
|
| 242 |
#. Confirm that your selected ranges correspond to the start points of the |
| 243 |
Forward and Backward EFA values (Steps 12-13). |
| 244 |
|
| 245 |
#. Confirm that your chi-squared plot is close to 1, without any major |
| 246 |
spikes. |
| 247 |
|
| 248 |
#. Confirm that your concentrations are not significantly altered by |
| 249 |
constraining the concentration to be positive (Steps 17-19). |
| 250 |
|
| 251 |
#. Click the “Save EFA Data (not profiles)” to save the EFA data, including the SVD, |
| 252 |
the Forward and Backward EFA data, the chi-squared, and the concentration, along |
| 253 |
with information about the selected ranges and the rotation method used. |
| 254 |
|
| 255 |
#. Click the “Done” button to send the scattering profiles to the Profiles Plot. |
| 256 |
|
| 257 |
#. In the main RAW window, go to the Profiles control tab and the Profiles plot. If |
| 258 |
it is not already, put the Profiles plot on a semi-Log or Log-Log scale. |
| 259 |
|
| 260 |
|efa_profiles_png| |
| 261 |
|
| 262 |
#. The three scattering profiles from EFA are in the manipulation list. The labels _0, |
| 263 |
_1, and _2 correspond to the 0, 1, and 2 components/ranges. |
| 264 |
|
| 265 |
* *Note:* Regardless of whether you use subtracted or unsubtracted data, these |
| 266 |
scattering profiles will be buffer subtracted, as the buffer represents a |
| 267 |
scattering component itself, and so (in theory) even if it is present will be |
| 268 |
separated out by successful EFA. |
| 269 |
|
| 270 |
|
| 271 |
Note: By default, RAW bins the profiles before doing an SVD and calculating the |
| 272 |
evolving factor plots, in order to speed up the process. The final EFA rotation is |
| 273 |
done on the full unbinned dataset. You can turn binning on and off and adjust |
| 274 |
the binning parameters in the Series options panel in the Advanced Options window. |
| 275 |
|
| 276 |
Note 2: If you save a pdf report of a series that has EFA analysis done on it, a |
| 277 |
summary of the EFA analysis and the various plots is saved in the report. |
| 278 |
|
| 279 |
.. |efa_series_plot_png| image:: images/efa_series_plot.png |
| 280 |
:target: ../_images/efa_series_plot.png |
| 281 |
|
| 282 |
.. |efa_panel_png| image:: images/efa_panel.png |
| 283 |
:target: ../_images/efa_panel.png |
| 284 |
|
| 285 |
.. |efa_components_png| image:: images/efa_components.png |
| 286 |
:target: ../_images/efa_components.png |
| 287 |
|
| 288 |
.. |efa_panel_2_png| image:: images/efa_panel_2.png |
| 289 |
:target: ../_images/efa_panel_2.png |
| 290 |
|
| 291 |
.. |efa_ranges_png| image:: images/efa_ranges.png |
| 292 |
:width: 200 px |
| 293 |
:target: ../_images/efa_ranges.png |
| 294 |
|
| 295 |
.. |efa_panel_3_png| image:: images/efa_panel_3.png |
| 296 |
:target: ../_images/efa_panel_3.png |
| 297 |
|
| 298 |
.. |efa_poor_rotation_png| image:: images/efa_poor_rotation.png |
| 299 |
:target: ../_images/efa_poor_rotation.png |
| 300 |
|
| 301 |
.. |efa_profiles_png| image:: images/efa_profiles.png |
| 302 |
:target: ../_images/efa_profiles.png |