| 1 |
=== Checklist for MDP release === |
| 2 |
|
| 3 |
- Check that new nodes have been explicitly imported in nodes/__init__.py and that they are listed in __all__: |
| 4 |
# create a list of defined nodes with: |
| 5 |
git grep 'class .*Node(' mdp/nodes | grep -v test | grep -v Scikits | cut -d ':' -f 2 | cut -d ' ' -f 2 | cut -d '(' -f 1 | sort > /tmp/list_defined |
| 6 |
# create a list of nodes imported in mdp.nodes with: |
| 7 |
python3 -c "import sys, mdp; [sys.stdout.write(i+'\n') for i in sorted([obj for obj in mdp.nodes.__dict__ if obj.endswith('Node') and not obj.endswith('ScikitsLearnNode')])]" > /tmp/list_in_dict |
| 8 |
# create a list of nodes in __all__ with: |
| 9 |
python3 -c "import sys, mdp; [sys.stdout.write(i+'\n') for i in sorted([obj for obj in mdp.nodes.__all__ if obj.endswith('Node') and not obj.endswith('ScikitsLearnNode')])]" > /tmp/list_in_all |
| 10 |
# compare those listsm for example with |
| 11 |
meld /tmp/list_defined /tmp/list_in_dict /tmp/list_in_all |
| 12 |
# These lists should be almost identical. Some nodes defined or found in the dict could be used only internally, |
| 13 |
# and thus not available through __all__, as of 2020.04.18 the only nodes that do not need to be in all three |
| 14 |
# lists are: |
| 15 |
- ICANode -> this is an internal node to be subclassed by nodes implementing some form of ICA |
| 16 |
- ProjectionNode -> this is an internal node used by XSFA |
| 17 |
- _ExpansionNode -> this is used internally by many SFA nodes) |
| 18 |
|
| 19 |
- Make sure that mdp/__init__.py has the right version number |
| 20 |
|
| 21 |
- Update date in variable __copyright__ in file mdp/__init__.py and in COPYRIGHT |
| 22 |
|
| 23 |
- Update CHANGES: you can generate a new bunch of CHANGES with: |
| 24 |
git log --no-color --pretty="format:%w(79,0,12)%ad: %s%+b" --date=short --no-merges --since=$LASTRELEASE |
| 25 |
|
| 26 |
where LASTRELEASE is the tag of the last release, e.g. MDP-3.6 |
| 27 |
|
| 28 |
You can then prepend the output of this command to the original CHANGES file, |
| 29 |
but *even better* would be to edit the result to only keep the changes |
| 30 |
that are relevant for the user like incompatibilities, new features, etc.. |
| 31 |
Not every commit is interesting in CHANGES: the exact commits are available in git. |
| 32 |
|
| 33 |
- Test that MDP can be installed with pip locally: |
| 34 |
# create a wheel |
| 35 |
python3 setup.py sdist bdist_wheel |
| 36 |
# create a new Python3 virtualenv |
| 37 |
python3 -m venv /tmp/mdp_test |
| 38 |
# activate the virtualenv |
| 39 |
source /tmp/mdp_test/bin/activate |
| 40 |
# install the wheel in the virtualenv |
| 41 |
pip install dist/XXX.whl |
| 42 |
# this should also install MDP hard-dependencies (numpy, future) |
| 43 |
# manually install MDP soft dependencies |
| 44 |
pip install scipy scikit-learn joblib libsvm pytest |
| 45 |
# change directory to be sure we are importing the pip-installed MDP |
| 46 |
cd ~ |
| 47 |
# now we should be able to import |
| 48 |
python -c 'import mdp; print(mdp.__file__, mdp.__version__)' |
| 49 |
# -> in the above verify that the file is in the clone of the MDP repo, |
| 50 |
# and that the version is correct |
| 51 |
# now we can run the tests |
| 52 |
python -c 'import mdp; mdp.test(); import bimdp; bimdp.test()' |
| 53 |
# -> verify that all soft depenencies are correctly detected by MDP, for example |
| 54 |
================================================== test session starts ============= |
| 55 |
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 |
| 56 |
python: 3.8.2.final.0 |
| 57 |
mdp: 3.6 |
| 58 |
parallel python: NOT AVAILABLE: No module named 'pp' |
| 59 |
shogun: NOT AVAILABLE: No module named 'shogun' |
| 60 |
libsvm: 3.23.0.4 |
| 61 |
joblib: 0.14.1 |
| 62 |
sklearn: 0.22.2.post1 |
| 63 |
numx: scipy 1.4.1 |
| 64 |
symeig: scipy.linalg.eigh |
| 65 |
Random Seed: 1343069334 |
| 66 |
|
| 67 |
rootdir: /tmp/mdp_test/lib/python3.8/site-packages/mdp/test |
| 68 |
# -> also, of course check that all tests are passing |
| 69 |
|
| 70 |
- create a release notes file (this is an edit version of the CHANGES file, |
| 71 |
it will be used as annotation for the git tag, and will be shown on GitHub in the |
| 72 |
release package) |
| 73 |
|
| 74 |
- tag release in git and use the release notes file as annotation for the tag |
| 75 |
git tag -a MDP-3.6 -F RELEASE_NOTES |
| 76 |
|
| 77 |
- push the tag git push --tags |
| 78 |
|
| 79 |
###### The stuff below is obsolete and needs to be updated ###### |
| 80 |
|
| 81 |
- generate tutorial, website, and API documentation [make website] |
| 82 |
- change homepage colors |
| 83 |
- short/long description should go: on SF.net description, tutorial, |
| 84 |
home page, modules __init__, software.incf.net. |
| 85 |
|
| 86 |
- update the package on PyPI: |
| 87 |
# first test on the test PyPI instance (you have to register there first) |
| 88 |
# - create a ~/.pypirc first: |
| 89 |
cat > ~/.pypirc <<EOF |
| 90 |
[distutils] |
| 91 |
index-servers= |
| 92 |
test |
| 93 |
|
| 94 |
[test] |
| 95 |
repository = https://testpypi.python.org/pypi |
| 96 |
username = otizonaizit |
| 97 |
EOF |
| 98 |
# - register |
| 99 |
python3 setup.py register -r test |
| 100 |
# go and check if it is there! |
| 101 |
# now use twine to upload and sign the packages |
| 102 |
twine upload -r test -s dist/MDP-3.4* |
| 103 |
# now create another virtualenv and try to test installation |
| 104 |
virtualenv mdp-27 && source /home/tiziano/mdp-27/bin/activate |
| 105 |
pip install -i https://testpypi.python.org/pypi MDP |
| 106 |
# if it fails because future or numpy are not on the test pypi server, |
| 107 |
# just install them with pip install without -i |
| 108 |
# |
| 109 |
# If all of the above works, you can finally upload to the real PyPI: |
| 110 |
# modify your ~/.pypirc: |
| 111 |
cat > ~/.pypirc <<EOF |
| 112 |
[distutils] |
| 113 |
index-servers= |
| 114 |
pypi |
| 115 |
test |
| 116 |
|
| 117 |
[test] |
| 118 |
repository = https://testpypi.python.org/pypi |
| 119 |
username = otizonaizit |
| 120 |
|
| 121 |
[pypi] |
| 122 |
repository = https://pypi.python.org/pypi |
| 123 |
username = otizonaizit |
| 124 |
EOF |
| 125 |
# now register |
| 126 |
python3 setup register |
| 127 |
# upload to pipy |
| 128 |
twine upload -s dist/MDP-3.4* |
| 129 |
# test that it works with pip |
| 130 |
virtualenv mdp-27 && source /home/tiziano/mdp-27/bin/activate |
| 131 |
pip install mdp |
| 132 |
# yeah, it works!!! |
| 133 |
- update on SF.net: |
| 134 |
release files: |
| 135 |
- sftp [email protected] |
| 136 |
- cd /home/frs/project/mdp-toolkit/mdp-toolkit/ |
| 137 |
- create a new directory for the release, for example for release 3.0: |
| 138 |
mkdir 3.0 |
| 139 |
cd 3.0 |
| 140 |
- upload the files there (note: the release notes should be named |
| 141 |
README.txt): |
| 142 |
file to upload are: .tar.gz, .zip, .exe, tutorial, release notes file |
| 143 |
- login to sourceforge, go to "Files" |
| 144 |
- select the new created directory |
| 145 |
- select the wheel file and set it as a default for windows and |
| 146 |
Mac by clicking on the "i" icon on the right, |
| 147 |
- select the tar.gz as a default for everything else |
| 148 |
- at that point the readme file should be automatically shown as release |
| 149 |
note file |
| 150 |
if README.txt is not shown, delete it and upload it through the web interface. |
| 151 |
make sure that it is shown. |
| 152 |
- more info: https://sourceforge.net/apps/trac/sourceforge/wiki/Release%20files%20for%20download |
| 153 |
- make the website within a clone of the docs repository with: |
| 154 |
- make website |
| 155 |
- be careful to read all warnings and messages, often things do not work as |
| 156 |
expected. |
| 157 |
- upload the pdf tutorial, which is in build/latex/MDP-tutorial.pdf, to sf.net |
| 158 |
as explained above for the source tarballs. |
| 159 |
- synchronize the site with: |
| 160 |
cd build/html |
| 161 |
rsync -av --delete-after . [email protected]:/home/project-web/mdp-toolkit/htdocs/ |
| 162 |
- more info: http://alexandria.wiki.sourceforge.net/Project+Web,+Shell,+VHOST+and+Database+Services |
| 163 |
- tag the docs repository: |
| 164 |
git tag -a MDP-3.5 |
| 165 |
git push --tags |
| 166 |
|
| 167 |
After release: |
| 168 |
- update version number in __init__ |
| 169 |
- send announcement to: |
| 170 |
connectionists: [email protected] |
| 171 |
ML-news: [email protected] |
| 172 |
numpy-discussion: [email protected] |
| 173 |
Scipy users: [email protected] |
| 174 |
mdp-users: [email protected] |
| 175 |
Python-announce: [email protected] |
| 176 |
|
| 177 |
- celebrate!! |
| 178 |
|