Debian Codegraph @

docs/source/changelog.rst

raw
1 .. This file is auto-converted from CHANGELOG.md (make update-changelog) -- do not edit
2
3 Change log
4 **********
5 1.3.1 (2026-02-01)
6 ==================
7
8 Bug Fixes
9 ---------
10
11 - Add typing_extensions to build system requirements (Python < 3.11).
12 `PR #7798 <https://github.com/datalad/datalad/pull/7798>`__ (by
13 `@lschr <https://github.com/lschr>`__)
14
15 - BF: Ensure annex-ignore config is immediately written for RIA stores.
16 Fixes `#5186 <https://github.com/datalad/datalad/issues/5186>`__ via
17 `PR #7799 <https://github.com/datalad/datalad/pull/7799>`__ (by
18 `@yarikoptic <https://github.com/yarikoptic>`__)
19
20 .. _section-1:
21
22 1.3.0 (2026-01-17)
23 ==================
24
25 Enhancements and New Features
26 -----------------------------
27
28 - maint: drop Python 3.9 - update CI and codebase. `PR
29 #7777 <https://github.com/datalad/datalad/pull/7777>`__ (by
30 `@yarikoptic <https://github.com/yarikoptic>`__)
31
32 .. _bug-fixes-1:
33
34 Bug Fixes
35 ---------
36
37 - fix: clone - do not run ‘git annex init’ if there is .noannex file.
38 Fixes `#7774 <https://github.com/datalad/datalad/issues/7774>`__ via
39 `PR #7775 <https://github.com/datalad/datalad/pull/7775>`__ (by
40 `@yarikoptic <https://github.com/yarikoptic>`__)
41
42 Documentation
43 -------------
44
45 - doc: Use importlib.metadata for author info in manpage generation to
46 avoid None’s and double nested <>. `PR
47 #7788 <https://github.com/datalad/datalad/pull/7788>`__ (by
48 `@yarikoptic <https://github.com/yarikoptic>`__)
49
50 Tests
51 -----
52
53 - Add testing and compatibility fixes for Python 3.14 support. `PR
54 #7784 <https://github.com/datalad/datalad/pull/7784>`__ (by
55 `@yarikoptic <https://github.com/yarikoptic>`__)
56
57 - BF(TST): Clean up provider config file in test_access_denied. `PR
58 #7786 <https://github.com/datalad/datalad/pull/7786>`__ (by
59 `@yarikoptic <https://github.com/yarikoptic>`__)
60
61 - ci: avoid running various test workflows on BOTH PR and branch
62 (unless master and maint). `PR
63 #7787 <https://github.com/datalad/datalad/pull/7787>`__ (by
64 `@yarikoptic <https://github.com/yarikoptic>`__)
65
66 .. _section-2:
67
68 1.2.3 (2025-10-30)
69 ==================
70
71 Dependencies
72 ------------
73
74 - Boost minimal git-annex version to 10.20230126. `PR
75 #7693 <https://github.com/datalad/datalad/pull/7693>`__ (by
76 `@yarikoptic <https://github.com/yarikoptic>`__)
77
78 .. _tests-1:
79
80 Tests
81 -----
82
83 - BF: altogether xfail the test_download_ftp – too flaky/unknown reason
84 ATM. `PR #7758 <https://github.com/datalad/datalad/pull/7758>`__ (by
85 `@yarikoptic <https://github.com/yarikoptic>`__)
86
87 .. _section-3:
88
89 1.2.2 (2025-10-15)
90 ==================
91
92 .. _bug-fixes-2:
93
94 Bug Fixes
95 ---------
96
97 - Remove absolute path specification from ``/bin/ls`` invocation while
98 interrogating remote path. `PR
99 #7730 <https://github.com/datalad/datalad/pull/7730>`__ (by
100 `@malikwirin <https://github.com/malikwirin>`__)
101
102 .. _documentation-1:
103
104 Documentation
105 -------------
106
107 - DOC: clarify in config documentation that we log to stderr not
108 stdout. `PR #7734 <https://github.com/datalad/datalad/pull/7734>`__
109 (by `@yarikoptic <https://github.com/yarikoptic>`__)
110
111 Internal
112 --------
113
114 - Move project metadata from setup.py to pyproject.toml. `PR
115 #7740 <https://github.com/datalad/datalad/pull/7740>`__ (by
116 `@lschr <https://github.com/lschr>`__)
117
118 .. _tests-2:
119
120 Tests
121 -----
122
123 - TST: Allow for tests under NFS (cron run only) for x20 more time. `PR
124 #7749 <https://github.com/datalad/datalad/pull/7749>`__ (by
125 `@yarikoptic <https://github.com/yarikoptic>`__)
126
127 - chore: Upgrade/switch back to ubuntu-latest for github CI and use
128 generalized neurodebian-ci-setup script (over https now). `PR
129 #7751 <https://github.com/datalad/datalad/pull/7751>`__ (by
130 `@yarikoptic <https://github.com/yarikoptic>`__)
131
132 - Add use of pytest-retry and mark test_download_ftp to be retried. `PR
133 #7754 <https://github.com/datalad/datalad/pull/7754>`__ (by
134 `@yarikoptic <https://github.com/yarikoptic>`__)
135
136 - Mark test_producer_future_key[10] as xfail on Windows. `PR
137 #7755 <https://github.com/datalad/datalad/pull/7755>`__ (by
138 `@yarikoptic <https://github.com/yarikoptic>`__)
139
140 .. _section-4:
141
142 1.2.1 (2025-07-02)
143 ==================
144
145 .. _bug-fixes-3:
146
147 Bug Fixes
148 ---------
149
150 - Resolve datetime library warnings. `PR
151 #7714 <https://github.com/datalad/datalad/pull/7714>`__ (by
152 `@emmanuel-ferdman <https://github.com/emmanuel-ferdman>`__)
153
154 - BF: Prevent infinite recursion in clone when a remote points to
155 itself. Fixes
156 `#7721 <https://github.com/datalad/datalad/issues/7721>`__ via `PR
157 #7731 <https://github.com/datalad/datalad/pull/7731>`__ (by
158 `@yarikoptic <https://github.com/yarikoptic>`__)
159
160 .. _section-5:
161
162 1.2.0 (2025-05-21)
163 ==================
164
165 .. _enhancements-and-new-features-1:
166
167 Enhancements and New Features
168 -----------------------------
169
170 - Remove import of GIT_SSH_COMMAND within .consts (import directly from
171 datalad.runner.gitrunner). `PR
172 #7656 <https://github.com/datalad/datalad/pull/7656>`__ (by
173 `@yarikoptic <https://github.com/yarikoptic>`__)
174
175 .. _internal-1:
176
177 Internal
178 --------
179
180 - Make github workflow which would ensure that a “release” PR against
181 ``master`` would have everything from ``maint`` merged. `PR
182 #7590 <https://github.com/datalad/datalad/pull/7590>`__ (by
183 `@yarikoptic <https://github.com/yarikoptic>`__)
184
185 .. _section-6:
186
187 1.1.6 (2025-05-18)
188 ==================
189
190 .. _documentation-2:
191
192 Documentation
193 -------------
194
195 - enh: update acknowledgments with additional grants. `PR
196 #7716 <https://github.com/datalad/datalad/pull/7716>`__ (by
197 `@yarikoptic <https://github.com/yarikoptic>`__)
198
199 .. _tests-3:
200
201 Tests
202 -----
203
204 - chore: for github tests CI - py 3.13 matrix should also (as for 3.12)
205 use most recent available git-annex. `PR
206 #7710 <https://github.com/datalad/datalad/pull/7710>`__ (by
207 `@yarikoptic <https://github.com/yarikoptic>`__)
208
209 .. _section-7:
210
211 1.1.5 (2024-12-15)
212 ==================
213
214 .. _tests-4:
215
216 Tests
217 -----
218
219 - test: xfail test_subsuperdataset_save on newer gits. `PR
220 #7687 <https://github.com/datalad/datalad/pull/7687>`__ (by
221 `@yarikoptic <https://github.com/yarikoptic>`__)
222
223 - test: refactor test_parallel.py to be abit more pytest’y. `PR
224 #7690 <https://github.com/datalad/datalad/pull/7690>`__ (by
225 `@yarikoptic <https://github.com/yarikoptic>`__)
226
227 - BF: use datalad/packages method of installing git-annex. `PR
228 #7692 <https://github.com/datalad/datalad/pull/7692>`__ (by
229 `@yarikoptic <https://github.com/yarikoptic>`__)
230
231 .. _section-8:
232
233 1.1.4 (2024-11-18)
234 ==================
235
236 .. _bug-fixes-4:
237
238 Bug Fixes
239 ---------
240
241 - Exit with the original non-0 exit code if underlying functionality,
242 in particular “datalad run”, returned incomplete result record with a
243 non-0 exit code. Fixes
244 `#7504 <https://github.com/datalad/datalad/issues/7504>`__ via `PR
245 #7641 <https://github.com/datalad/datalad/pull/7641>`__ (by
246 `@yarikoptic <https://github.com/yarikoptic>`__)
247
248 - Provide detail on why CHECKURL failed for datalad and archive special
249 remotes (would require new AnnexRemote release (above 1.6.5) to take
250 advantage of this PR). `PR
251 #7648 <https://github.com/datalad/datalad/pull/7648>`__ (by
252 `@yarikoptic <https://github.com/yarikoptic>`__)
253
254 - BF: allow for empty output directory to be specified to run. Fixes
255 `#7653 <https://github.com/datalad/datalad/issues/7653>`__ via `PR
256 #7654 <https://github.com/datalad/datalad/pull/7654>`__ (by
257 `@yarikoptic <https://github.com/yarikoptic>`__)
258
259 - fix: do load extension interfaces if early parsing errors out. Fixes
260 `#7678 <https://github.com/datalad/datalad/issues/7678>`__ via `PR
261 #7679 <https://github.com/datalad/datalad/pull/7679>`__ (by
262 `@yarikoptic <https://github.com/yarikoptic>`__)
263
264 - Drop Python 3.8. Fixes
265 `#7678 <https://github.com/datalad/datalad/issues/7678>`__ via `PR
266 #7682 <https://github.com/datalad/datalad/pull/7682>`__ (by
267 `@yarikoptic <https://github.com/yarikoptic>`__)
268
269 Performance
270 -----------
271
272 - OPT: use set O(log(n)) instead of list O(n) for checking if modified
273 in \_check_files. `PR
274 #7655 <https://github.com/datalad/datalad/pull/7655>`__ (by
275 `@yarikoptic <https://github.com/yarikoptic>`__)
276
277 .. _tests-5:
278
279 Tests
280 -----
281
282 - Declare minimal compat version of pytest to be 7.0. Fixes
283 `#7555 <https://github.com/datalad/datalad/issues/7555>`__ via `PR
284 #7645 <https://github.com/datalad/datalad/pull/7645>`__ (by
285 `@yarikoptic <https://github.com/yarikoptic>`__)
286
287 - BF: replace url for codecov uploader for macos to versioned one with
288 two archs. Fixes
289 `#7642 <https://github.com/datalad/datalad/issues/7642>`__ via `PR
290 #7649 <https://github.com/datalad/datalad/pull/7649>`__ (by
291 `@yarikoptic <https://github.com/yarikoptic>`__)
292
293 .. _section-9:
294
295 1.1.3 (2024-08-08)
296 ==================
297
298 .. _tests-6:
299
300 Tests
301 -----
302
303 - Account for the fix in git-annex behavior in
304 test_add_delete_after_and_drop_subdir. `PR
305 #7640 <https://github.com/datalad/datalad/pull/7640>`__ (by
306 `@yarikoptic <https://github.com/yarikoptic>`__)
307
308 .. _section-10:
309
310 1.1.2 (2024-07-25)
311 ==================
312
313 .. _bug-fixes-5:
314
315 Bug Fixes
316 ---------
317
318 - Correct remote OS detection when working with RIA (ORA) stores: this
319 should enable RIA operations, including push, from Mac clients to
320 Linux hosts (and likely vice versa). Fixes
321 `#7536 <https://github.com/datalad/datalad/issues/7536>`__ via `PR
322 #7549 <https://github.com/datalad/datalad/pull/7549>`__ (by
323 `@mslw <https://github.com/mslw>`__)
324
325 - Allow only one thread in S3 downloader’s progress report callback.
326 `PR #7636 <https://github.com/datalad/datalad/pull/7636>`__ (by
327 `@christian-monch <https://github.com/christian-monch>`__)
328
329 .. _section-11:
330
331 1.1.1 (2024-07-03)
332 ==================
333
334 .. _bug-fixes-6:
335
336 Bug Fixes
337 ---------
338
339 - Ensure timestamps of files in ZIP archives are within years
340 1980-2107. Fixes
341 `#3753 <https://github.com/datalad/datalad/issues/3753>`__ via `PR
342 #7450 <https://github.com/datalad/datalad/pull/7450>`__ (by
343 `@adswa <https://github.com/adswa>`__)
344
345 .. _documentation-3:
346
347 Documentation
348 -------------
349
350 - Update README.md: improve wording. `PR
351 #7550 <https://github.com/datalad/datalad/pull/7550>`__ (by
352 `@alliesw <https://github.com/alliesw>`__)
353
354 .. _internal-2:
355
356 Internal
357 --------
358
359 - Add codespell and minor fixuppers to pre-commit configuration and
360 apply it to non-``datalad/`` components. `PR
361 #7621 <https://github.com/datalad/datalad/pull/7621>`__ (by
362 `@yarikoptic <https://github.com/yarikoptic>`__)
363
364 .. _tests-7:
365
366 Tests
367 -----
368
369 - For appveyor ssh setup, setup MaxSessions 100 to avoid ‘channel 22:
370 open failed: connect failed: open failed’. `PR
371 #7617 <https://github.com/datalad/datalad/pull/7617>`__ (by
372 `@yarikoptic <https://github.com/yarikoptic>`__)
373
374 - test_gracefull_death: raise test_gracefull_death threshold to 300
375 from 100. `PR #7619 <https://github.com/datalad/datalad/pull/7619>`__
376 (by `@yarikoptic <https://github.com/yarikoptic>`__)
377
378 - Make test for presence of max_path in partitions not run for current
379 psutil 6.0.0. `PR
380 #7622 <https://github.com/datalad/datalad/pull/7622>`__ (by
381 `@yarikoptic <https://github.com/yarikoptic>`__)
382
383 .. _section-12:
384
385 1.1.0 (2024-06-06)
386 ==================
387
388 .. _dependencies-1:
389
390 Dependencies
391 ------------
392
393 - Deprecated ``boto`` is replaced with ``boto3`` (used to handle AWS S3
394 downloads). Fixes
395 `#5597 <https://github.com/datalad/datalad/issues/5597>`__ via `PR
396 #7340 <https://github.com/datalad/datalad/pull/7340>`__ (by
397 `@mslw <https://github.com/mslw>`__,
398 `@effigies <https://github.com/effigies>`__, and
399 `@yarikoptic <https://github.com/yarikoptic>`__). Remaining issues:
400
401 - no download progress indication,
402 - no “Range” support (for partial downloads).
403
404 .. _internal-3:
405
406 Internal
407 --------
408
409 - Retry logic for S3 connections is now handed over to Boto3 and its
410 standard mode, removing our custom method. `PR
411 #7340 <https://github.com/datalad/datalad/pull/7340>`__
412
413 .. _section-13:
414
415 1.0.3 (2024-06-06)
416 ==================
417
418 .. _bug-fixes-7:
419
420 Bug Fixes
421 ---------
422
423 - Raise exception if an annex remote process without console tries to
424 interact with the user, e.g. prompt for a password. `PR
425 #7578 <https://github.com/datalad/datalad/pull/7578>`__ (by
426 `@christian-monch <https://github.com/christian-monch>`__)
427
428 - Fix add-archive-content for patool>=2.0. `PR
429 #7603 <https://github.com/datalad/datalad/pull/7603>`__ (by
430 `@dguibert <https://github.com/dguibert>`__)
431
432 .. _internal-4:
433
434 Internal
435 --------
436
437 - Fixup minor typos in documentation/comments using fresh codespell.
438 `PR #7610 <https://github.com/datalad/datalad/pull/7610>`__ (by
439 `@yarikoptic <https://github.com/yarikoptic>`__)
440
441 .. _tests-8:
442
443 Tests
444 -----
445
446 - Stop testing on Python 3.7. Switch MacOS tests to 3.11, include 3.11
447 in Appveyor, and use 3.8 for other tests. Fixes
448 `#7584 <https://github.com/datalad/datalad/issues/7584>`__ via `PR
449 #7585 <https://github.com/datalad/datalad/pull/7585>`__ (by
450 `@mslw <https://github.com/mslw>`__)
451
452 - Convert ``.travis.yml`` to GitHub Actions workflow. Fixes
453 `#7574 <https://github.com/datalad/datalad/issues/7574>`__ via `PR
454 #7600 <https://github.com/datalad/datalad/pull/7600>`__ (by
455 `@jwodder <https://github.com/jwodder>`__)
456
457 - Cancel lengthy running workflows if a new commit is pushed. `PR
458 #7601 <https://github.com/datalad/datalad/pull/7601>`__ (by
459 `@jwodder <https://github.com/jwodder>`__)
460
461 .. _section-14:
462
463 1.0.2 (2024-04-19)
464 ==================
465
466 .. _tests-9:
467
468 Tests
469 -----
470
471 - Relax condition in ``test_force_checkdatapresent`` to avoid flaky
472 test failures. `PR
473 #7581 <https://github.com/datalad/datalad/pull/7581>`__ (by
474 `@christian-monch <https://github.com/christian-monch>`__)
475
476 .. _section-15:
477
478 1.0.1 (2024-04-17)
479 ==================
480
481 .. _internal-5:
482
483 Internal
484 --------
485
486 - The main entrypoint for annex remotes now also runs the standard
487 extension load hook. This enables extensions to alter annex remote
488 implementation behavior in the same way than other DataLad
489 components. (by `@mih <https://github.com/mih>`__)
490
491 .. _section-16:
492
493 1.0.0 (2024-04-06)
494 ==================
495
496 Breaking Changes
497 ----------------
498
499 - Merging maint to make the first major release. `PR
500 #7577 <https://github.com/datalad/datalad/pull/7577>`__ (by
501 `@yarikoptic <https://github.com/yarikoptic>`__)
502
503 .. _enhancements-and-new-features-2:
504
505 Enhancements and New Features
506 -----------------------------
507
508 - Increase minimal Git version to 2.25. Fixes
509 `#7389 <https://github.com/datalad/datalad/issues/7389>`__ via `PR
510 #7431 <https://github.com/datalad/datalad/pull/7431>`__ (by
511 `@adswa <https://github.com/adswa>`__)
512
513 .. _section-17:
514
515 0.19.6 (2024-02-02)
516 ===================
517
518 .. _enhancements-and-new-features-3:
519
520 Enhancements and New Features
521 -----------------------------
522
523 - Add the “http_token” authentication mechanism which provides
524 ‘Authentication: Token {TOKEN}’ header. `PR
525 #7551 <https://github.com/datalad/datalad/pull/7551>`__ (by
526 `@yarikoptic <https://github.com/yarikoptic>`__)
527
528 .. _internal-6:
529
530 Internal
531 --------
532
533 - Update ``pytest_ignore_collect()`` for pytest 8.0. `PR
534 #7546 <https://github.com/datalad/datalad/pull/7546>`__ (by
535 `@jwodder <https://github.com/jwodder>`__)
536
537 - Add manual triggering support/documentation for release workflow. `PR
538 #7553 <https://github.com/datalad/datalad/pull/7553>`__ (by
539 `@yarikoptic <https://github.com/yarikoptic>`__)
540
541 .. _section-18:
542
543 0.19.5 (2023-12-28)
544 ===================
545
546 .. _tests-10:
547
548 Tests
549 -----
550
551 - Fix text to account for a recent change in git-annex dropping
552 sub-second clock precision. As a result we might not report push of
553 git-annex branch since there would be none. `PR
554 #7544 <https://github.com/datalad/datalad/pull/7544>`__ (by
555 `@yarikoptic <https://github.com/yarikoptic>`__)
556
557 .. _section-19:
558
559 0.19.4 (2023-12-13)
560 ===================
561
562 .. _bug-fixes-8:
563
564 Bug Fixes
565 ---------
566
567 - Update target detection for adjusted mode datasets has been improved.
568 Fixes `#7507 <https://github.com/datalad/datalad/issues/7507>`__ via
569 `PR #7522 <https://github.com/datalad/datalad/pull/7522>`__ (by
570 `@mih <https://github.com/mih>`__)
571
572 - Fix typos found by new codespell 2.2.6 and also add checking/fixing
573 “hidden files”. `PR
574 #7530 <https://github.com/datalad/datalad/pull/7530>`__ (by
575 `@yarikoptic <https://github.com/yarikoptic>`__)
576
577 .. _documentation-4:
578
579 Documentation
580 -------------
581
582 - Improve threaded-runner documentation. Fixes
583 `#7498 <https://github.com/datalad/datalad/issues/7498>`__ via `PR
584 #7500 <https://github.com/datalad/datalad/pull/7500>`__ (by
585 `@christian-monch <https://github.com/christian-monch>`__)
586
587 .. _internal-7:
588
589 Internal
590 --------
591
592 - add RRID to package metadata. `PR
593 #7495 <https://github.com/datalad/datalad/pull/7495>`__ (by
594 `@adswa <https://github.com/adswa>`__)
595
596 - Fix time_diff\* and time_remove benchmarks to account for long RFed
597 interfaces. `PR
598 #7502 <https://github.com/datalad/datalad/pull/7502>`__ (by
599 `@yarikoptic <https://github.com/yarikoptic>`__)
600
601 .. _tests-11:
602
603 Tests
604 -----
605
606 - Cache value of the has_symlink_capability to spare some cycles. `PR
607 #7471 <https://github.com/datalad/datalad/pull/7471>`__ (by
608 `@yarikoptic <https://github.com/yarikoptic>`__)
609
610 - RF(TST): use setup_method and teardown_method in
611 TestAddArchiveOptions. `PR
612 #7488 <https://github.com/datalad/datalad/pull/7488>`__ (by
613 `@yarikoptic <https://github.com/yarikoptic>`__)
614
615 - Announce test_clone_datasets_root xfail on github osx. `PR
616 #7489 <https://github.com/datalad/datalad/pull/7489>`__ (by
617 `@yarikoptic <https://github.com/yarikoptic>`__)
618
619 - Inform asv that there should be no warmup runs for time_remove
620 benchmark. `PR
621 #7505 <https://github.com/datalad/datalad/pull/7505>`__ (by
622 `@yarikoptic <https://github.com/yarikoptic>`__)
623
624 - BF(TST): Relax matching of git-annex error message about unsafe drop,
625 which was changed in 10.20231129-18-gfd0b510573. `PR
626 #7541 <https://github.com/datalad/datalad/pull/7541>`__ (by
627 `@yarikoptic <https://github.com/yarikoptic>`__)
628
629 .. _section-20:
630
631 0.19.3 (2023-08-10)
632 ===================
633
634 .. _bug-fixes-9:
635
636 Bug Fixes
637 ---------
638
639 - Type annotate get_status_dict and note that we can pass Exception or
640 CapturedException which is not subclass. `PR
641 #7403 <https://github.com/datalad/datalad/pull/7403>`__ (by
642 `@yarikoptic <https://github.com/yarikoptic>`__)
643
644 - BF: create-sibling-gitlab used to raise a TypeError when attempting a
645 recursive operation in a dataset with uninstalled subdatasets. It now
646 raises an impossible result instead. `PR
647 #7430 <https://github.com/datalad/datalad/pull/7430>`__ (by
648 `@adswa <https://github.com/adswa>`__)
649
650 - Pass branch option into recursive call within Install - for the cases
651 whenever install is invoked with URL(s). Fixes
652 `#7461 <https://github.com/datalad/datalad/issues/7461>`__ via `PR
653 #7463 <https://github.com/datalad/datalad/pull/7463>`__ (by
654 `@yarikoptic <https://github.com/yarikoptic>`__)
655
656 - Allow for reckless=ephemeral clone using relative path for the
657 original location. Fixes
658 `#7469 <https://github.com/datalad/datalad/issues/7469>`__ via `PR
659 #7472 <https://github.com/datalad/datalad/pull/7472>`__ (by
660 `@yarikoptic <https://github.com/yarikoptic>`__)
661
662 .. _documentation-5:
663
664 Documentation
665 -------------
666
667 - Fix a property name and default costs described in “getting
668 subdatasets” section of ``get`` documentation. Fixes
669 `#7458 <https://github.com/datalad/datalad/issues/7458>`__ via `PR
670 #7460 <https://github.com/datalad/datalad/pull/7460>`__ (by
671 `@mslw <https://github.com/mslw>`__)
672
673 .. _internal-8:
674
675 Internal
676 --------
677
678 - Copy an adjusted environment only if requested to do so. `PR
679 #7399 <https://github.com/datalad/datalad/pull/7399>`__ (by
680 `@christian-monch <https://github.com/christian-monch>`__)
681
682 - Eliminate uses of ``pkg_resources``. Fixes
683 `#7435 <https://github.com/datalad/datalad/issues/7435>`__ via `PR
684 #7439 <https://github.com/datalad/datalad/pull/7439>`__ (by
685 `@jwodder <https://github.com/jwodder>`__)
686
687 .. _tests-12:
688
689 Tests
690 -----
691
692 - Disable some S3 tests of their VCR taping where they fail for known
693 issues. `PR #7467 <https://github.com/datalad/datalad/pull/7467>`__
694 (by `@yarikoptic <https://github.com/yarikoptic>`__)
695
696 .. _section-21:
697
698 0.19.2 (2023-07-03)
699 ===================
700
701 .. _bug-fixes-10:
702
703 Bug Fixes
704 ---------
705
706 - Remove surrounding quotes in output filenames even for newer version
707 of annex. Fixes
708 `#7440 <https://github.com/datalad/datalad/issues/7440>`__ via `PR
709 #7443 <https://github.com/datalad/datalad/pull/7443>`__ (by
710 `@yarikoptic <https://github.com/yarikoptic>`__)
711
712 .. _documentation-6:
713
714 Documentation
715 -------------
716
717 - DOC: clarify description of the “install” interface to reflect its
718 convoluted behavior. `PR
719 #7445 <https://github.com/datalad/datalad/pull/7445>`__ (by
720 `@yarikoptic <https://github.com/yarikoptic>`__)
721
722 .. _section-22:
723
724 0.19.1 (2023-06-26)
725 ===================
726
727 .. _internal-9:
728
729 Internal
730 --------
731
732 - Make compatible with upcoming release of git-annex (next after
733 10.20230407) and pass explicit core.quotepath=false to all git calls.
734 Also added ``tools/find-hanged-tests`` helper. `PR
735 #7372 <https://github.com/datalad/datalad/pull/7372>`__ (by
736 `@yarikoptic <https://github.com/yarikoptic>`__)
737
738 .. _tests-13:
739
740 Tests
741 -----
742
743 - Adjust tests for upcoming release of git-annex (next after
744 10.20230407) and ignore DeprecationWarning for pkg_resources for now.
745 `PR #7372 <https://github.com/datalad/datalad/pull/7372>`__ (by
746 `@yarikoptic <https://github.com/yarikoptic>`__)
747
748 .. _section-23:
749
750 0.19.0 (2023-06-14)
751 ===================
752
753 .. _enhancements-and-new-features-4:
754
755 Enhancements and New Features
756 -----------------------------
757
758 - Address gitlab API special character restrictions. `PR
759 #7407 <https://github.com/datalad/datalad/pull/7407>`__ (by
760 `@jsheunis <https://github.com/jsheunis>`__)
761
762 - BF: The default layout of create-sibling-gitlab is now
763 ``collection``. The previous default, ``hierarchy`` has been removed
764 as it failed in –recursive mode in different edgecases. For
765 single-level datasets, the outcome of ``collection`` and
766 ``hierarchy`` is identical. `PR
767 #7410 <https://github.com/datalad/datalad/pull/7410>`__ (by
768 `@jsheunis <https://github.com/jsheunis>`__ and
769 `@adswa <https://github.com/adswa>`__)
770
771 .. _bug-fixes-11:
772
773 Bug Fixes
774 ---------
775
776 - WTF - bring back and extend information on metadata extractors etc,
777 and allow for sections to have subsections and be selected at both
778 levels `PR #7309 <https://github.com/datalad/datalad/pull/7309>`__
779 (by `@yarikoptic <https://github.com/yarikoptic>`__)
780
781 - BF: Run an actual git invocation with interactive commit config. `PR
782 #7398 <https://github.com/datalad/datalad/pull/7398>`__ (by
783 `@adswa <https://github.com/adswa>`__)
784
785 .. _dependencies-2:
786
787 Dependencies
788 ------------
789
790 - Raise minimal version of tqdm (progress bars) to v.4.32.0 `PR
791 #7330 <https://github.com/datalad/datalad/pull/7330>`__ (by
792 `@mslw <https://github.com/mslw>`__)
793
794 .. _documentation-7:
795
796 Documentation
797 -------------
798
799 - DOC: Add a “User messaging” design doc. `PR
800 #7310 <https://github.com/datalad/datalad/pull/7310>`__ (by
801 `@jsheunis <https://github.com/jsheunis>`__)
802
803 .. _tests-14:
804
805 Tests
806 -----
807
808 - Remove nose-based testing utils and possibility to test extensions
809 using nose. `PR
810 #7261 <https://github.com/datalad/datalad/pull/7261>`__ (by
811 `@yarikoptic <https://github.com/yarikoptic>`__)
812
813 .. _section-24:
814
815 0.18.5 (2023-06-13)
816 ===================
817
818 .. _bug-fixes-12:
819
820 Bug Fixes
821 ---------
822
823 - More correct summary reporting for relaxed (no size) –annex. `PR
824 #7050 <https://github.com/datalad/datalad/pull/7050>`__ (by
825 `@yarikoptic <https://github.com/yarikoptic>`__)
826
827 - ENH: minor tune up of addurls to be more tolerant and “informative”.
828 `PR #7388 <https://github.com/datalad/datalad/pull/7388>`__ (by
829 `@yarikoptic <https://github.com/yarikoptic>`__)
830
831 - Ensure that data generated by timeout handlers in the asynchronous
832 runner are accessible via the result generator, even if no other
833 other events occur. `PR
834 #7390 <https://github.com/datalad/datalad/pull/7390>`__ (by
835 `@christian-monch <https://github.com/christian-monch>`__)
836
837 - Do not map (leave as is) trailing / or  in github URLs. `PR
838 #7418 <https://github.com/datalad/datalad/pull/7418>`__ (by
839 `@yarikoptic <https://github.com/yarikoptic>`__)
840
841 .. _documentation-8:
842
843 Documentation
844 -------------
845
846 - Use ``sphinx_autodoc_typehints``. Fixes
847 `#7404 <https://github.com/datalad/datalad/issues/7404>`__ via `PR
848 #7412 <https://github.com/datalad/datalad/pull/7412>`__ (by
849 `@jwodder <https://github.com/jwodder>`__)
850
851 .. _internal-10:
852
853 Internal
854 --------
855
856 - Discontinue ConfigManager abuse for Git identity warning. `PR
857 #7378 <https://github.com/datalad/datalad/pull/7378>`__ (by
858 `@mih <https://github.com/mih>`__) and `PR
859 #7392 <https://github.com/datalad/datalad/pull/7392>`__ (by
860 `@yarikoptic <https://github.com/yarikoptic>`__)
861
862 .. _tests-15:
863
864 Tests
865 -----
866
867 - Boost python to 3.8 during extensions testing. `PR
868 #7413 <https://github.com/datalad/datalad/pull/7413>`__ (by
869 `@yarikoptic <https://github.com/yarikoptic>`__)
870
871 - Skip test_system_ssh_version if no ssh found + split parsing into
872 separate test. `PR
873 #7422 <https://github.com/datalad/datalad/pull/7422>`__ (by
874 `@yarikoptic <https://github.com/yarikoptic>`__)
875
876 .. _section-25:
877
878 0.18.4 (2023-05-16)
879 ===================
880
881 .. _bug-fixes-13:
882
883 Bug Fixes
884 ---------
885
886 - Provider config files were ignored, when CWD changed between
887 different datasets during runtime. Fixes
888 `#7347 <https://github.com/datalad/datalad/issues/7347>`__ via `PR
889 #7357 <https://github.com/datalad/datalad/pull/7357>`__ (by
890 `@bpoldrack <https://github.com/bpoldrack>`__)
891
892 .. _documentation-9:
893
894 Documentation
895 -------------
896
897 - Added a workaround for an issue with documentation theme (search
898 function not working on Read the Docs). Fixes
899 `#7374 <https://github.com/datalad/datalad/issues/7374>`__ via `PR
900 #7385 <https://github.com/datalad/datalad/pull/7385>`__ (by
901 `@mslw <https://github.com/mslw>`__)
902
903 .. _internal-11:
904
905 Internal
906 --------
907
908 - Type-annotate ``datalad/support/gitrepo.py``. `PR
909 #7341 <https://github.com/datalad/datalad/pull/7341>`__ (by
910 `@jwodder <https://github.com/jwodder>`__)
911
912 .. _tests-16:
913
914 Tests
915 -----
916
917 - Fix failing testing on CI `PR
918 #7379 <https://github.com/datalad/datalad/pull/7379>`__ (by
919 `@yarikoptic <https://github.com/yarikoptic>`__)
920
921 - use sample S3 url DANDI archive,
922 - use our copy of old .deb from datasets.datalad.org instead of
923 snapshots.d.o
924 - use specific miniconda installer for py 3.7.
925
926 .. _section-26:
927
928 0.18.3 (2023-03-25)
929 ===================
930
931 .. _bug-fixes-14:
932
933 Bug Fixes
934 ---------
935
936 - Fixed that the ``get`` command would fail, when subdataset
937 source-candidate-templates where using the ``path`` property from
938 ``.gitmodules``. Also enhance the respective documentation for the
939 ``get`` command. Fixes
940 `#7274 <https://github.com/datalad/datalad/issues/7274>`__ via `PR
941 #7280 <https://github.com/datalad/datalad/pull/7280>`__ (by
942 `@bpoldrack <https://github.com/bpoldrack>`__)
943
944 - Improve up-to-dateness of config reports across manager instances.
945 Fixes `#7299 <https://github.com/datalad/datalad/issues/7299>`__ via
946 `PR #7301 <https://github.com/datalad/datalad/pull/7301>`__ (by
947 `@mih <https://github.com/mih>`__)
948
949 - BF: GitRepo.merge do not allow merging unrelated unconditionally. `PR
950 #7312 <https://github.com/datalad/datalad/pull/7312>`__ (by
951 `@yarikoptic <https://github.com/yarikoptic>`__)
952
953 - Do not render (empty) WTF report on other records. `PR
954 #7322 <https://github.com/datalad/datalad/pull/7322>`__ (by
955 `@yarikoptic <https://github.com/yarikoptic>`__)
956
957 - Fixed a bug where changing DataLad’s log level could lead to failing
958 git-annex calls. Fixes
959 `#7328 <https://github.com/datalad/datalad/issues/7328>`__ via `PR
960 #7329 <https://github.com/datalad/datalad/pull/7329>`__ (by
961 `@bpoldrack <https://github.com/bpoldrack>`__)
962
963 - Fix an issue with uninformative error reporting by the datalad
964 special remote. Fixes
965 `#7332 <https://github.com/datalad/datalad/issues/7332>`__ via `PR
966 #7333 <https://github.com/datalad/datalad/pull/7333>`__ (by
967 `@bpoldrack <https://github.com/bpoldrack>`__)
968
969 - Fix save to not force committing into git if reference dataset is
970 pure git (not git-annex). Fixes
971 `#7351 <https://github.com/datalad/datalad/issues/7351>`__ via `PR
972 #7355 <https://github.com/datalad/datalad/pull/7355>`__ (by
973 `@yarikoptic <https://github.com/yarikoptic>`__)
974
975 .. _documentation-10:
976
977 Documentation
978 -------------
979
980 - Include a few previously missing commands in html API docs. Fixes
981 `#7288 <https://github.com/datalad/datalad/issues/7288>`__ via `PR
982 #7289 <https://github.com/datalad/datalad/pull/7289>`__ (by
983 `@mslw <https://github.com/mslw>`__)
984
985 .. _internal-12:
986
987 Internal
988 --------
989
990 - Type-annotate almost all of ``datalad/utils.py``; add
991 ``datalad/typing.py``. `PR
992 #7317 <https://github.com/datalad/datalad/pull/7317>`__ (by
993 `@jwodder <https://github.com/jwodder>`__)
994
995 - Type-annotate and fix ``datalad/support/strings.py``. `PR
996 #7318 <https://github.com/datalad/datalad/pull/7318>`__ (by
997 `@jwodder <https://github.com/jwodder>`__)
998
999 - Type-annotate ``datalad/support/globbedpaths.py``. `PR
1000 #7327 <https://github.com/datalad/datalad/pull/7327>`__ (by
1001 `@jwodder <https://github.com/jwodder>`__)
1002
1003 - Extend type-annotations for ``datalad/support/path.py``. `PR
1004 #7336 <https://github.com/datalad/datalad/pull/7336>`__ (by
1005 `@jwodder <https://github.com/jwodder>`__)
1006
1007 - Type-annotate various things in ``datalad/runner/``. `PR
1008 #7337 <https://github.com/datalad/datalad/pull/7337>`__ (by
1009 `@jwodder <https://github.com/jwodder>`__)
1010
1011 - Type-annotate some more files in ``datalad/support/``. `PR
1012 #7339 <https://github.com/datalad/datalad/pull/7339>`__ (by
1013 `@jwodder <https://github.com/jwodder>`__)
1014
1015 .. _tests-17:
1016
1017 Tests
1018 -----
1019
1020 - Skip or xfail some currently failing or stalling tests. `PR
1021 #7331 <https://github.com/datalad/datalad/pull/7331>`__ (by
1022 `@yarikoptic <https://github.com/yarikoptic>`__)
1023
1024 - Skip with_sameas_remote when rsync and annex are incompatible. Fixes
1025 `#7320 <https://github.com/datalad/datalad/issues/7320>`__ via `PR
1026 #7342 <https://github.com/datalad/datalad/pull/7342>`__ (by
1027 `@bpoldrack <https://github.com/bpoldrack>`__)
1028
1029 - Fix testing assumption - do create pure GitRepo superdataset and test
1030 against it. `PR
1031 #7353 <https://github.com/datalad/datalad/pull/7353>`__ (by
1032 `@yarikoptic <https://github.com/yarikoptic>`__)
1033
1034 .. _section-27:
1035
1036 0.18.2 (2023-02-27)
1037 ===================
1038
1039 .. _bug-fixes-15:
1040
1041 Bug Fixes
1042 ---------
1043
1044 - Fix ``create-sibling`` for non-English SSH remotes by providing
1045 ``LC_ALL=C`` for the ``ls`` call. `PR
1046 #7265 <https://github.com/datalad/datalad/pull/7265>`__ (by
1047 `@nobodyinperson <https://github.com/nobodyinperson>`__)
1048
1049 - Fix EnsureListOf() and EnsureTupleOf() for string inputs. `PR
1050 #7267 <https://github.com/datalad/datalad/pull/7267>`__ (by
1051 `@nobodyinperson <https://github.com/nobodyinperson>`__)
1052
1053 - create-sibling: Use C.UTF-8 locale instead of C on the remote end.
1054 `PR #7273 <https://github.com/datalad/datalad/pull/7273>`__ (by
1055 `@nobodyinperson <https://github.com/nobodyinperson>`__)
1056
1057 - Address compatibility with most recent git-annex where info would
1058 exit with non-0. `PR
1059 #7292 <https://github.com/datalad/datalad/pull/7292>`__ (by
1060 `@yarikoptic <https://github.com/yarikoptic>`__)
1061
1062 .. _dependencies-3:
1063
1064 Dependencies
1065 ------------
1066
1067 - Revert “Revert”Remove chardet version upper limit”“. `PR
1068 #7263 <https://github.com/datalad/datalad/pull/7263>`__ (by
1069 `@yarikoptic <https://github.com/yarikoptic>`__)
1070
1071 .. _internal-13:
1072
1073 Internal
1074 --------
1075
1076 - Codespell more (CHANGELOGs etc) and remove custom CLI options from
1077 tox.ini. `PR #7271 <https://github.com/datalad/datalad/pull/7271>`__
1078 (by `@yarikoptic <https://github.com/yarikoptic>`__)
1079
1080 .. _tests-18:
1081
1082 Tests
1083 -----
1084
1085 - Use older python 3.8 in testing nose utils in github-action
1086 test-nose. Fixes
1087 `#7259 <https://github.com/datalad/datalad/issues/7259>`__ via `PR
1088 #7260 <https://github.com/datalad/datalad/pull/7260>`__ (by
1089 `@yarikoptic <https://github.com/yarikoptic>`__)
1090
1091 .. _section-28:
1092
1093 0.18.1 (2023-01-16)
1094 ===================
1095
1096 .. _bug-fixes-16:
1097
1098 Bug Fixes
1099 ---------
1100
1101 - Fixes crashes on windows where DataLad was mistaking git-annex
1102 10.20221212 for a not yet released git-annex version and trying to
1103 use a new feature. Fixes
1104 `#7248 <https://github.com/datalad/datalad/issues/7248>`__ via `PR
1105 #7249 <https://github.com/datalad/datalad/pull/7249>`__ (by
1106 `@bpoldrack <https://github.com/bpoldrack>`__)
1107
1108 .. _documentation-11:
1109
1110 Documentation
1111 -------------
1112
1113 - DOC: fix EnsureCallable docstring. `PR
1114 #7245 <https://github.com/datalad/datalad/pull/7245>`__ (by
1115 `@matrss <https://github.com/matrss>`__)
1116
1117 .. _performance-1:
1118
1119 Performance
1120 -----------
1121
1122 - Integrate buffer size optimization from datalad-next, leading to
1123 significant performance improvement for status and diff. Fixes
1124 `#7190 <https://github.com/datalad/datalad/issues/7190>`__ via `PR
1125 #7250 <https://github.com/datalad/datalad/pull/7250>`__ (by
1126 `@bpoldrack <https://github.com/bpoldrack>`__)
1127
1128 .. _section-29:
1129
1130 0.18.0 (2022-12-31)
1131 ===================
1132
1133 .. _breaking-changes-1:
1134
1135 Breaking Changes
1136 ----------------
1137
1138 - Move all old-style metadata commands ``aggregate_metadata``,
1139 ``search``, ``metadata`` and ``extract-metadata``, as well as the
1140 ``cfg_metadatatypes`` procedure and the old metadata extractors into
1141 the datalad-deprecated extension. Now recommended way of handling
1142 metadata is to install the datalad-metalad extension instead. Fixes
1143 `#7012 <https://github.com/datalad/datalad/issues/7012>`__ via `PR
1144 #7014 <https://github.com/datalad/datalad/pull/7014>`__
1145
1146 - Automatic reconfiguration of the ORA special remote when cloning from
1147 RIA stores now only applies locally rather than being committed. `PR
1148 #7235 <https://github.com/datalad/datalad/pull/7235>`__ (by
1149 `@bpoldrack <https://github.com/bpoldrack>`__)
1150
1151 .. _enhancements-and-new-features-5:
1152
1153 Enhancements and New Features
1154 -----------------------------
1155
1156 - A repository description can be specified with a new
1157 ``--description`` option when creating siblings using
1158 ``create-sibling-[gin|gitea|github|gogs]``. Fixes
1159 `#6816 <https://github.com/datalad/datalad/issues/6816>`__ via `PR
1160 #7109 <https://github.com/datalad/datalad/pull/7109>`__ (by
1161 `@mslw <https://github.com/mslw>`__)
1162
1163 - Make validation failure of alternative constraints more informative.
1164 Fixes `#7092 <https://github.com/datalad/datalad/issues/7092>`__ via
1165 `PR #7132 <https://github.com/datalad/datalad/pull/7132>`__ (by
1166 `@bpoldrack <https://github.com/bpoldrack>`__)
1167
1168 - Saving removed dataset content was sped-up, and reporting of types of
1169 removed content now accurately states ``dataset`` for added and
1170 removed subdatasets, instead of ``file``. Moreover, saving previously
1171 staged deletions is now also reported. `PR
1172 #6784 <https://github.com/datalad/datalad/pull/6784>`__ (by
1173 `@mih <https://github.com/mih>`__)
1174
1175 - ``foreach-dataset`` command got a new possible value for the
1176 –output-streamns\|–o-s option ‘relpath’ to capture and pass-through
1177 prefixing with path to subds. Very handy for e.g. running
1178 ``git grep`` command across subdatasets. `PR
1179 #7071 <https://github.com/datalad/datalad/pull/7071>`__ (by
1180 `@yarikoptic <https://github.com/yarikoptic>`__)
1181
1182 - New config
1183 ``datalad.create-sibling-ghlike.extra-remote-settings.NETLOC.KEY=VALUE``
1184 allows to add and/or overwrite local configuration for the created
1185 sibling by the commands
1186 ``create-sibling-<gin|gitea|github|gitlab|gogs>``. `PR
1187 #7213 <https://github.com/datalad/datalad/pull/7213>`__ (by
1188 `@matrss <https://github.com/matrss>`__)
1189
1190 - The ``siblings`` command does not concern the user with messages
1191 about inconsequential failure to annex-enable a remote anymore. `PR
1192 #7217 <https://github.com/datalad/datalad/pull/7217>`__ (by
1193 `@bpoldrack <https://github.com/bpoldrack>`__)
1194
1195 - ORA special remote now allows to override its configuration locally.
1196 `PR #7235 <https://github.com/datalad/datalad/pull/7235>`__ (by
1197 `@bpoldrack <https://github.com/bpoldrack>`__)
1198
1199 - Added a ‘ria’ special remote to provide backwards compatibility with
1200 datasets that were set up with the deprecated
1201 `ria-remote <https://github.com/datalad/git-annex-ria-remote>`__. `PR
1202 #7235 <https://github.com/datalad/datalad/pull/7235>`__ (by
1203 `@bpoldrack <https://github.com/bpoldrack>`__)
1204
1205 .. _bug-fixes-17:
1206
1207 Bug Fixes
1208 ---------
1209
1210 - When ``create-sibling-ria`` was invoked with a sibling name of a
1211 pre-existing sibling, a duplicate key in the result record caused a
1212 crashed. Fixes
1213 `#6950 <https://github.com/datalad/datalad/issues/6950>`__ via `PR
1214 #6952 <https://github.com/datalad/datalad/pull/6952>`__ (by
1215 `@adswa <https://api.github.com/users/adswa>`__)
1216
1217 .. _documentation-12:
1218
1219 Documentation
1220 -------------
1221
1222 - create-sibling-ria’s docstring now defines the schema of RIA URLs and
1223 clarifies internal layout of a RIA store. `PR
1224 #6861 <https://github.com/datalad/datalad/pull/6861>`__ (by
1225 `@adswa <https://api.github.com/users/adswa>`__)
1226
1227 - Move maintenance team info from issue to CONTRIBUTING. `PR
1228 #6904 <https://github.com/datalad/datalad/pull/6904>`__ (by
1229 `@adswa <https://api.github.com/users/adswa>`__)
1230
1231 - Describe specifications for a DataLad GitHub Action. `PR
1232 #6931 <https://github.com/datalad/datalad/pull/6931>`__ (by
1233 `@thewtex <https://api.github.com/users/thewtex>`__)
1234
1235 - Fix capitalization of some service names. `PR
1236 #6936 <https://github.com/datalad/datalad/pull/6936>`__ (by
1237 `@aqw <https://api.github.com/users/aqw>`__)
1238
1239 - Command categories in help text are more consistently named. `PR
1240 #7027 <https://github.com/datalad/datalad/pull/7027>`__ (by
1241 `@aqw <https://api.github.com/users/aqw>`__)
1242
1243 - DOC: Add design document on Tests and CI. `PR
1244 #7195 <https://github.com/datalad/datalad/pull/7195>`__ (by
1245 `@adswa <https://github.com/adswa>`__)
1246
1247 - CONTRIBUTING.md was extended with up-to-date information on CI
1248 logging, changelog and release procedures. `PR
1249 #7204 <https://github.com/datalad/datalad/pull/7204>`__ (by
1250 `@yarikoptic <https://github.com/yarikoptic>`__)
1251
1252 .. _internal-14:
1253
1254 Internal
1255 --------
1256
1257 - Allow EnsureDataset constraint to handle Path instances. Fixes
1258 `#7069 <https://github.com/datalad/datalad/issues/7069>`__ via `PR
1259 #7133 <https://github.com/datalad/datalad/pull/7133>`__ (by
1260 `@bpoldrack <https://github.com/bpoldrack>`__)
1261
1262 - Use ``looseversion.LooseVersion`` as drop-in replacement for
1263 ``distutils.version.LooseVersion`` Fixes
1264 `#6307 <https://github.com/datalad/datalad/issues/6307>`__ via `PR
1265 #6839 <https://github.com/datalad/datalad/pull/6839>`__ (by
1266 `@effigies <https://api.github.com/users/effigies>`__)
1267
1268 - Use –pathspec-from-file where possible instead of passing long lists
1269 of paths to git/git-annex calls. Fixes
1270 `#6922 <https://github.com/datalad/datalad/issues/6922>`__ via `PR
1271 #6932 <https://github.com/datalad/datalad/pull/6932>`__ (by
1272 `@yarikoptic <https://api.github.com/users/yarikoptic>`__)
1273
1274 - Make clone_dataset() better patchable ny extensions and less
1275 monolithic. `PR
1276 #7017 <https://github.com/datalad/datalad/pull/7017>`__ (by
1277 `@mih <https://api.github.com/users/mih>`__)
1278
1279 - Remove ``simplejson`` in favor of using ``json``. Fixes
1280 `#7034 <https://github.com/datalad/datalad/issues/7034>`__ via `PR
1281 #7035 <https://github.com/datalad/datalad/pull/7035>`__ (by
1282 `@christian-monch <https://api.github.com/users/christian-monch>`__)
1283
1284 - Fix an error in the command group names-test. `PR
1285 #7044 <https://github.com/datalad/datalad/pull/7044>`__ (by
1286 `@christian-monch <https://api.github.com/users/christian-monch>`__)
1287
1288 - Move eval_results() into interface.base to simplify imports for
1289 command implementations. Deprecate use from interface.utils
1290 accordingly. Fixes
1291 `#6694 <https://github.com/datalad/datalad/issues/6694>`__ via `PR
1292 #7170 <https://github.com/datalad/datalad/pull/7170>`__ (by
1293 `@adswa <https://github.com/adswa>`__)
1294
1295 .. _performance-2:
1296
1297 Performance
1298 -----------
1299
1300 - Use regular dicts instead of OrderedDicts for speedier operations.
1301 Fixes `#6566 <https://github.com/datalad/datalad/issues/6566>`__ via
1302 `PR #7174 <https://github.com/datalad/datalad/pull/7174>`__ (by
1303 `@adswa <https://github.com/adswa>`__)
1304
1305 - Reimplement ``get_submodules_()`` without ``get_content_info()`` for
1306 substantial performance boosts especially for large datasets with few
1307 subdatasets. Originally proposed in `PR
1308 #6942 <https://github.com/datalad/datalad/pull/6942>`__ by
1309 `@mih <https://github.com/mih>`__, fixing
1310 `#6940 <https://github.com/datalad/datalad/issues/6940>`__. `PR
1311 #7189 <https://github.com/datalad/datalad/pull/7189>`__ (by
1312 `@adswa <https://github.com/adswa>`__). Complemented with `PR
1313 #7220 <https://github.com/datalad/datalad/pull/7220>`__ (by
1314 `@yarikoptic <https://github.com/yarikoptic>`__) to avoid ``O(N^2)``
1315 (instead of ``O(N*log(N))`` performance in some cases.
1316
1317 - Use –include=\* or –anything instead of –copies 0 to speed up
1318 get_content_annexinfo. `PR
1319 #7230 <https://github.com/datalad/datalad/pull/7230>`__ (by
1320 `@yarikoptic <https://github.com/yarikoptic>`__)
1321
1322 .. _tests-19:
1323
1324 Tests
1325 -----
1326
1327 - Re-enable two now-passing core test on Windows CI. `PR
1328 #7152 <https://github.com/datalad/datalad/pull/7152>`__ (by
1329 `@adswa <https://api.github.com/users/adswa>`__)
1330
1331 - Remove the ``with_testrepos`` decorator and associated tests for it
1332 Fixes `#6752 <https://github.com/datalad/datalad/issues/6752>`__ via
1333 `PR #7176 <https://github.com/datalad/datalad/pull/7176>`__ (by
1334 `@adswa <https://api.github.com/users/adswa>`__)
1335
1336 .. _section-30:
1337
1338 0.17.10 (2022-12-14)
1339 ====================
1340
1341 .. _enhancements-and-new-features-6:
1342
1343 Enhancements and New Features
1344 -----------------------------
1345
1346 - Enhance concurrent invocation behavior of ``ThreadedRunner.run()``.
1347 If possible invocations are serialized instead of raising *re-enter*
1348 runtime errors. Deadlock situations are detected and runtime errors
1349 are raised instead of deadlocking. Fixes
1350 `#7138 <https://github.com/datalad/datalad/issues/7138>`__ via `PR
1351 #7201 <https://github.com/datalad/datalad/pull/7201>`__ (by
1352 `@christian-monch <https://github.com/christian-monch>`__)
1353
1354 - Exceptions bubbling up through CLI are now reported on including
1355 their chain of **cause**. Fixes
1356 `#7163 <https://github.com/datalad/datalad/issues/7163>`__ via `PR
1357 #7210 <https://github.com/datalad/datalad/pull/7210>`__ (by
1358 `@bpoldrack <https://github.com/bpoldrack>`__)
1359
1360 .. _bug-fixes-18:
1361
1362 Bug Fixes
1363 ---------
1364
1365 - BF: read RIA config from stdin instead of temporary file. Fixes
1366 `#6514 <https://github.com/datalad/datalad/issues/6514>`__ via `PR
1367 #7147 <https://github.com/datalad/datalad/pull/7147>`__ (by
1368 `@adswa <https://github.com/adswa>`__)
1369
1370 - Prevent doomed annex calls on files we already know are untracked.
1371 Fixes `#7032 <https://github.com/datalad/datalad/issues/7032>`__ via
1372 `PR #7166 <https://github.com/datalad/datalad/pull/7166>`__ (by
1373 `@adswa <https://github.com/adswa>`__)
1374
1375 - Comply to Posix-like clone URL formats on Windows. Fixes
1376 `#7180 <https://github.com/datalad/datalad/issues/7180>`__ via `PR
1377 #7181 <https://github.com/datalad/datalad/pull/7181>`__ (by
1378 `@adswa <https://github.com/adswa>`__)
1379
1380 - Ensure that paths used in the datalad-url field of .gitmodules are
1381 posix. Fixes
1382 `#7182 <https://github.com/datalad/datalad/issues/7182>`__ via `PR
1383 #7183 <https://github.com/datalad/datalad/pull/7183>`__ (by
1384 `@adswa <https://github.com/adswa>`__)
1385
1386 - Bandaids for export-to-figshare to restore functionality. `PR
1387 #7188 <https://github.com/datalad/datalad/pull/7188>`__ (by
1388 `@adswa <https://github.com/adswa>`__)
1389
1390 - Fixes hanging threads when ``close()`` or ``del`` where called in
1391 ``BatchedCommand`` instances. That could lead to hanging tests if the
1392 tests used the ``@serve_path_via_http()``-decorator Fixes
1393 `#6804 <https://github.com/datalad/datalad/issues/6804>`__ via `PR
1394 #7201 <https://github.com/datalad/datalad/pull/7201>`__ (by
1395 `@christian-monch <https://github.com/christian-monch>`__)
1396
1397 - Interpret file-URL path components according to the local operating
1398 system as described in RFC 8089. With this fix,
1399 ``datalad.network.RI('file:...').localpath`` returns a correct local
1400 path on Windows if the RI is constructed with a file-URL. Fixes
1401 `#7186 <https://github.com/datalad/datalad/issues/7186>`__ via `PR
1402 #7206 <https://github.com/datalad/datalad/pull/7206>`__ (by
1403 `@christian-monch <https://github.com/christian-monch>`__)
1404
1405 - Fix a bug when retrieving several files from a RIA store via SSH,
1406 when the annex key does not contain size information. Fixes
1407 `#7214 <https://github.com/datalad/datalad/issues/7214>`__ via `PR
1408 #7215 <https://github.com/datalad/datalad/pull/7215>`__ (by
1409 `@mslw <https://github.com/mslw>`__)
1410
1411 - Interface-specific (python vs CLI) doc generation for commands and
1412 their parameters was broken when brackets were used within the
1413 interface markups. Fixes
1414 `#7225 <https://github.com/datalad/datalad/issues/7225>`__ via `PR
1415 #7226 <https://github.com/datalad/datalad/pull/7226>`__ (by
1416 `@bpoldrack <https://github.com/bpoldrack>`__)
1417
1418 .. _documentation-13:
1419
1420 Documentation
1421 -------------
1422
1423 - Fix documentation of ``Runner.run()`` to not accept strings. Instead,
1424 encoding must be ensured by the caller. Fixes
1425 `#7145 <https://github.com/datalad/datalad/issues/7145>`__ via `PR
1426 #7155 <https://github.com/datalad/datalad/pull/7155>`__ (by
1427 `@bpoldrack <https://github.com/bpoldrack>`__)
1428
1429 .. _internal-15:
1430
1431 Internal
1432 --------
1433
1434 - Fix import of the ``ls`` command from datalad-deprecated for
1435 benchmarks. Fixes
1436 `#7149 <https://github.com/datalad/datalad/issues/7149>`__ via `PR
1437 #7154 <https://github.com/datalad/datalad/pull/7154>`__ (by
1438 `@bpoldrack <https://github.com/bpoldrack>`__)
1439
1440 - Unify definition of parameter choices with ``datalad clean``. Fixes
1441 `#7026 <https://github.com/datalad/datalad/issues/7026>`__ via `PR
1442 #7161 <https://github.com/datalad/datalad/pull/7161>`__ (by
1443 `@bpoldrack <https://github.com/bpoldrack>`__)
1444
1445 .. _tests-20:
1446
1447 Tests
1448 -----
1449
1450 - Fix test failure with old annex. Fixes
1451 `#7157 <https://github.com/datalad/datalad/issues/7157>`__ via `PR
1452 #7159 <https://github.com/datalad/datalad/pull/7159>`__ (by
1453 `@bpoldrack <https://github.com/bpoldrack>`__)
1454
1455 - Re-enable now passing test_path_diff test on Windows. Fixes
1456 `#3725 <https://github.com/datalad/datalad/issues/3725>`__ via `PR
1457 #7194 <https://github.com/datalad/datalad/pull/7194>`__ (by
1458 `@yarikoptic <https://github.com/yarikoptic>`__)
1459
1460 - Use Plaintext keyring backend in tests to avoid the need for
1461 (interactive) authentication to unlock the keyring during (CI-) test
1462 runs. Fixes
1463 `#6623 <https://github.com/datalad/datalad/issues/6623>`__ via `PR
1464 #7209 <https://github.com/datalad/datalad/pull/7209>`__ (by
1465 `@bpoldrack <https://github.com/bpoldrack>`__)
1466
1467 .. _section-31:
1468
1469 0.17.9 (2022-11-07)
1470 ===================
1471
1472 .. _bug-fixes-19:
1473
1474 Bug Fixes
1475 ---------
1476
1477 - Various small fixups ran after looking post-release and trying to
1478 build Debian package. `PR
1479 #7112 <https://github.com/datalad/datalad/pull/7112>`__ (by
1480 `@yarikoptic <https://github.com/yarikoptic>`__)
1481
1482 - BF: Fix add-archive-contents try-finally statement by defining
1483 variable earlier. `PR
1484 #7117 <https://github.com/datalad/datalad/pull/7117>`__ (by
1485 `@adswa <https://github.com/adswa>`__)
1486
1487 - Fix RIA file URL reporting in exception handling. `PR
1488 #7123 <https://github.com/datalad/datalad/pull/7123>`__ (by
1489 `@adswa <https://github.com/adswa>`__)
1490
1491 - HTTP download treated ‘429 - too many requests’ as an authentication
1492 issue and was consequently trying to obtain credentials. Fixes
1493 `#7129 <https://github.com/datalad/datalad/issues/7129>`__ via `PR
1494 #7129 <https://github.com/datalad/datalad/pull/7129>`__ (by
1495 `@bpoldrack <https://github.com/bpoldrack>`__)
1496
1497 .. _dependencies-4:
1498
1499 Dependencies
1500 ------------
1501
1502 - Unrestrict pytest and pytest-cov versions. `PR
1503 #7125 <https://github.com/datalad/datalad/pull/7125>`__ (by
1504 `@jwodder <https://github.com/jwodder>`__)
1505
1506 - Remove remaining references to ``nose`` and the implied requirement
1507 for building the documentation Fixes
1508 `#7100 <https://github.com/datalad/datalad/issues/7100>`__ via `PR
1509 #7136 <https://github.com/datalad/datalad/pull/7136>`__ (by
1510 `@bpoldrack <https://github.com/bpoldrack>`__)
1511
1512 .. _internal-16:
1513
1514 Internal
1515 --------
1516
1517 - Use datalad/release-action. Fixes
1518 `#7110 <https://github.com/datalad/datalad/issues/7110>`__. `PR
1519 #7111 <https://github.com/datalad/datalad/pull/7111>`__ (by
1520 `@jwodder <https://github.com/jwodder>`__)
1521
1522 - Fix all logging to use %-interpolation and not .format, sort imports
1523 in touched files, add pylint-ing for % formatting in log messages to
1524 ``tox -e lint``. `PR
1525 #7118 <https://github.com/datalad/datalad/pull/7118>`__ (by
1526 `@yarikoptic <https://github.com/yarikoptic>`__)
1527
1528 .. _tests-21:
1529
1530 Tests
1531 -----
1532
1533 - Increase the upper time limit after which we assume that a process is
1534 stalling. That should reduce false positives from
1535 ``datalad.support.tests.test_parallel.py::test_stalling``, without
1536 impacting the runtime of passing tests. `PR
1537 #7119 <https://github.com/datalad/datalad/pull/7119>`__ (by
1538 `@christian-monch <https://github.com/christian-monch>`__)
1539
1540 - XFAIL a check on length of results in test_gracefull_death. `PR
1541 #7126 <https://github.com/datalad/datalad/pull/7126>`__ (by
1542 `@yarikoptic <https://github.com/yarikoptic>`__)
1543
1544 - Configure Git to allow for “file” protocol in tests. `PR
1545 #7130 <https://github.com/datalad/datalad/pull/7130>`__ (by
1546 `@yarikoptic <https://github.com/yarikoptic>`__)
1547
1548 .. _section-32:
1549
1550 0.17.8 (2022-10-24)
1551 ===================
1552
1553 .. _bug-fixes-20:
1554
1555 Bug Fixes
1556 ---------
1557
1558 - Prevent adding duplicate entries to .gitmodules. `PR
1559 #7088 <https://github.com/datalad/datalad/pull/7088>`__ (by
1560 `@yarikoptic <https://github.com/yarikoptic>`__)
1561
1562 - [BF] Prevent double yielding of impossible get result Fixes
1563 `#5537 <https://github.com/datalad/datalad/issues/5537>`__. `PR
1564 #7093 <https://github.com/datalad/datalad/pull/7093>`__ (by
1565 `@jsheunis <https://github.com/jsheunis>`__)
1566
1567 - Stop rendering the output of internal ``subdatset()`` call in the
1568 results of ``run_procedure()``. Fixes
1569 `#7091 <https://github.com/datalad/datalad/issues/7091>`__ via `PR
1570 #7094 <https://github.com/datalad/datalad/pull/7094>`__ (by
1571 `@mslw <https://github.com/mslw>`__ &
1572 `@mih <https://github.com/mih>`__)
1573
1574 - Improve handling of ``--existing reconfigure`` in
1575 ``create-sibling-ria``: previously, the command would not make the
1576 underlying ``git init`` call for existing local repositories, leading
1577 to some configuration updates not being applied. Partially addresses
1578 https://github.com/datalad/datalad/issues/6967 via
1579 https://github.com/datalad/datalad/pull/7095 (by @mslw)
1580
1581 - Ensure subprocess environments have a valid path in
1582 ``os.environ['PWD']``, even if a Path-like object was given to the
1583 runner on subprocess creation or invocation. Fixes
1584 `#7040 <https://github.com/datalad/datalad/issues/7040>`__ via `PR
1585 #7107 <https://github.com/datalad/datalad/pull/7107>`__ (by
1586 `@christian-monch <https://github.com/christian-monch>`__)
1587
1588 - Improved reporting when using ``dry-run`` with github-like
1589 ``create-sibling*`` commands (``-gin``, ``-gitea``, ``-github``,
1590 ``-gogs``). The result messages will now display names of the
1591 repositories which would be created (useful for recursive
1592 operations). `PR
1593 #7103 <https://github.com/datalad/datalad/pull/7103>`__ (by
1594 `@mslw <https://github.com/mslw>`__)
1595
1596 .. _section-33:
1597
1598 0.17.7 (2022-10-14)
1599 ===================
1600
1601 .. _bug-fixes-21:
1602
1603 Bug Fixes
1604 ---------
1605
1606 - Let ``EnsureChoice`` report the value is failed validating. `PR
1607 #7067 <https://github.com/datalad/datalad/pull/7067>`__ (by
1608 `@mih <https://github.com/mih>`__)
1609
1610 - Avoid writing to stdout/stderr from within datalad sshrun. This could
1611 lead to broken pipe errors when cloning via SSH and was superfluous
1612 to begin with. Fixes https://github.com/datalad/datalad/issues/6599
1613 via https://github.com/datalad/datalad/pull/7072 (by @bpoldrack)
1614
1615 - BF: lock across threads check/instantiation of Flyweight instances.
1616 Fixes `#6598 <https://github.com/datalad/datalad/issues/6598>`__ via
1617 `PR #7075 <https://github.com/datalad/datalad/pull/7075>`__ (by
1618 `@yarikoptic <https://github.com/yarikoptic>`__)
1619
1620 .. _internal-17:
1621
1622 Internal
1623 --------
1624
1625 - Do not use ``gen4``-metadata methods in ``datalad metadata``-command.
1626 `PR #7001 <https://github.com/datalad/datalad/pull/7001>`__ (by
1627 `@christian-monch <https://github.com/christian-monch>`__)
1628
1629 - Revert “Remove chardet version upper limit” (introduced in
1630 0.17.6~11^2) to bring back upper limit <= 5.0.0 on chardet. Otherwise
1631 we can get some deprecation warnings from requests `PR
1632 #7057 <https://github.com/datalad/datalad/pull/7057>`__ (by
1633 `@yarikoptic <https://github.com/yarikoptic>`__)
1634
1635 - Ensure that ``BatchedCommandError`` is raised if the subprocesses of
1636 ``BatchedCommand`` fails or raises a ``CommandError``. `PR
1637 #7068 <https://github.com/datalad/datalad/pull/7068>`__ (by
1638 `@christian-monch <https://github.com/christian-monch>`__)
1639
1640 - RF: remove unused code str-ing PurePath. `PR
1641 #7073 <https://github.com/datalad/datalad/pull/7073>`__ (by
1642 `@yarikoptic <https://github.com/yarikoptic>`__)
1643
1644 - Update GitHub Actions action versions. `PR
1645 #7082 <https://github.com/datalad/datalad/pull/7082>`__ (by
1646 `@jwodder <https://github.com/jwodder>`__)
1647
1648 .. _tests-22:
1649
1650 Tests
1651 -----
1652
1653 - Fix broken test helpers for result record testing that would falsely
1654 pass. `PR #7002 <https://github.com/datalad/datalad/pull/7002>`__ (by
1655 `@bpoldrack <https://github.com/bpoldrack>`__)
1656
1657 .. _section-34:
1658
1659 0.17.6 (2022-09-21)
1660 ===================
1661
1662 .. _bug-fixes-22:
1663
1664 Bug Fixes
1665 ---------
1666
1667 - UX: push - provide specific error with details if push failed due to
1668 permission issue. `PR
1669 #7011 <https://github.com/datalad/datalad/pull/7011>`__ (by
1670 `@yarikoptic <https://github.com/yarikoptic>`__)
1671
1672 - Fix datalad –help to not have *Global options* empty with python 3.10
1673 and list options in “options:” section. `PR
1674 #7028 <https://github.com/datalad/datalad/pull/7028>`__ (by
1675 `@yarikoptic <https://github.com/yarikoptic>`__)
1676
1677 - Let ``create`` touch the dataset root, if not saving in parent
1678 dataset. `PR #7036 <https://github.com/datalad/datalad/pull/7036>`__
1679 (by `@mih <https://github.com/mih>`__)
1680
1681 - Let ``get_status_dict()`` use exception message if none is passed.
1682 `PR #7037 <https://github.com/datalad/datalad/pull/7037>`__ (by
1683 `@mih <https://github.com/mih>`__)
1684
1685 - Make choices for ``status|diff --annex`` and
1686 ``status|diff --untracked`` visible. `PR
1687 #7039 <https://github.com/datalad/datalad/pull/7039>`__ (by
1688 `@mih <https://github.com/mih>`__)
1689
1690 - push: Assume 0 bytes pushed if git-annex does not provide bytesize.
1691 `PR #7049 <https://github.com/datalad/datalad/pull/7049>`__ (by
1692 `@yarikoptic <https://github.com/yarikoptic>`__)
1693
1694 .. _internal-18:
1695
1696 Internal
1697 --------
1698
1699 - Use scriv for CHANGELOG generation in release workflow. `PR
1700 #7009 <https://github.com/datalad/datalad/pull/7009>`__ (by
1701 `@jwodder <https://github.com/jwodder>`__)
1702
1703 - Stop using auto. `PR
1704 #7024 <https://github.com/datalad/datalad/pull/7024>`__ (by
1705 `@jwodder <https://github.com/jwodder>`__)
1706
1707 .. _tests-23:
1708
1709 Tests
1710 -----
1711
1712 - Allow for any 2 from first 3 to be consumed in test_gracefull_death.
1713 `PR #7041 <https://github.com/datalad/datalad/pull/7041>`__ (by
1714 `@yarikoptic <https://github.com/yarikoptic>`__)
1715
1716 --------------
1717
1718 0.17.5 (Fri Sep 02 2022)
1719 ========================
1720
1721 Bug Fix
1722 -------
1723
1724 - BF: blacklist 23.9.0 of keyring as introduces regression
1725 `#7003 <https://github.com/datalad/datalad/pull/7003>`__
1726 (`@yarikoptic <https://github.com/yarikoptic>`__)
1727 - Make the manpages build reproducible via datalad.source.epoch (to be
1728 used in Debian packaging)
1729 `#6997 <https://github.com/datalad/datalad/pull/6997>`__
1730 (`@lamby <https://github.com/lamby>`__ [email protected]
1731 `@yarikoptic <https://github.com/yarikoptic>`__)
1732 - BF: backquote path/drive in Changelog
1733 `#6997 <https://github.com/datalad/datalad/pull/6997>`__
1734 (`@yarikoptic <https://github.com/yarikoptic>`__)
1735
1736 Authors: 3
1737 ----------
1738
1739 - Chris Lamb (`@lamby <https://github.com/lamby>`__)
1740 - DataLad Bot ([email protected])
1741 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
1742
1743 --------------
1744
1745 0.17.4 (Tue Aug 30 2022)
1746 ========================
1747
1748 .. _bug-fix-1:
1749
1750 Bug Fix
1751 -------
1752
1753 - BF: make logic more consistent for files=[] argument (which is False
1754 but not None)
1755 `#6976 <https://github.com/datalad/datalad/pull/6976>`__
1756 (`@yarikoptic <https://github.com/yarikoptic>`__)
1757 - Run pytests in parallel (-n 2) on appveyor
1758 `#6987 <https://github.com/datalad/datalad/pull/6987>`__
1759 (`@yarikoptic <https://github.com/yarikoptic>`__)
1760 - Add workflow for autogenerating changelog snippets
1761 `#6981 <https://github.com/datalad/datalad/pull/6981>`__
1762 (`@jwodder <https://github.com/jwodder>`__)
1763 - Provide ``/dev/null`` (``b:\nul`` on Windows) instead of empty string
1764 as a git-repo to avoid reading local repo configuration
1765 `#6986 <https://github.com/datalad/datalad/pull/6986>`__
1766 (`@yarikoptic <https://github.com/yarikoptic>`__)
1767 - RF: call_from_parser - move code into “else” to simplify reading etc
1768 `#6982 <https://github.com/datalad/datalad/pull/6982>`__
1769 (`@yarikoptic <https://github.com/yarikoptic>`__)
1770 - BF: if early attempt to parse resulted in error, setup subparsers
1771 `#6980 <https://github.com/datalad/datalad/pull/6980>`__
1772 (`@yarikoptic <https://github.com/yarikoptic>`__)
1773 - Run pytests in parallel (-n 2) on Travis
1774 `#6915 <https://github.com/datalad/datalad/pull/6915>`__
1775 (`@yarikoptic <https://github.com/yarikoptic>`__)
1776 - Send one character (no newline) to stdout in protocol test to
1777 guarantee a single “message” and thus a single custom value
1778 `#6978 <https://github.com/datalad/datalad/pull/6978>`__
1779 (`@christian-monch <https://github.com/christian-monch>`__)
1780
1781 .. _tests-24:
1782
1783 Tests
1784 -----
1785
1786 - TST: test_stalling – wait x10 not just x5 time
1787 `#6995 <https://github.com/datalad/datalad/pull/6995>`__
1788 (`@yarikoptic <https://github.com/yarikoptic>`__)
1789
1790 .. _authors-3-1:
1791
1792 Authors: 3
1793 ----------
1794
1795 - Christian Mnch
1796 (`@christian-monch <https://github.com/christian-monch>`__)
1797 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
1798 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
1799
1800 --------------
1801
1802 0.17.3 (Tue Aug 23 2022)
1803 ========================
1804
1805 .. _bug-fix-2:
1806
1807 Bug Fix
1808 -------
1809
1810 - BF: git_ignore_check do not overload possible value of stdout/err if
1811 present `#6937 <https://github.com/datalad/datalad/pull/6937>`__
1812 (`@yarikoptic <https://github.com/yarikoptic>`__)
1813 - DOCfix: fix docstring GeneratorStdOutErrCapture to say that treats
1814 both stdout and stderr identically
1815 `#6930 <https://github.com/datalad/datalad/pull/6930>`__
1816 (`@yarikoptic <https://github.com/yarikoptic>`__)
1817 - Explain purpose of create-sibling-ria’s –post-update-hook
1818 `#6958 <https://github.com/datalad/datalad/pull/6958>`__
1819 (`@mih <https://github.com/mih>`__)
1820 - ENH+BF: get_parent_paths - make / into sep option and consistently
1821 use “/” as path separator
1822 `#6963 <https://github.com/datalad/datalad/pull/6963>`__
1823 (`@yarikoptic <https://github.com/yarikoptic>`__)
1824 - BF(TEMP): use git-annex from neurodebian -devel to gain fix for bug
1825 detected with datalad-crawler
1826 `#6965 <https://github.com/datalad/datalad/pull/6965>`__
1827 (`@yarikoptic <https://github.com/yarikoptic>`__)
1828 - BF(TST): make tests use *path* helper for Windows “friendliness” of
1829 the tests `#6955 <https://github.com/datalad/datalad/pull/6955>`__
1830 (`@yarikoptic <https://github.com/yarikoptic>`__)
1831 - BF(TST): prevent auto-upgrade of “remote” test sibling, do not use
1832 local path for URL
1833 `#6957 <https://github.com/datalad/datalad/pull/6957>`__
1834 (`@yarikoptic <https://github.com/yarikoptic>`__)
1835 - Forbid drop operation from symlink’ed annex (e.g. due to being cloned
1836 with –reckless=ephemeral) to prevent data-loss
1837 `#6959 <https://github.com/datalad/datalad/pull/6959>`__
1838 (`@mih <https://github.com/mih>`__)
1839 - Acknowledge git-config comment chars
1840 `#6944 <https://github.com/datalad/datalad/pull/6944>`__
1841 (`@mih <https://github.com/mih>`__
1842 `@yarikoptic <https://github.com/yarikoptic>`__)
1843 - Minor tuneups to please updated codespell
1844 `#6956 <https://github.com/datalad/datalad/pull/6956>`__
1845 (`@yarikoptic <https://github.com/yarikoptic>`__)
1846 - TST: Add a testcase for #6950
1847 `#6957 <https://github.com/datalad/datalad/pull/6957>`__
1848 (`@adswa <https://github.com/adswa>`__)
1849 - BF+ENH(TST): fix typo in code of wtf filesystems reports
1850 `#6920 <https://github.com/datalad/datalad/pull/6920>`__
1851 (`@yarikoptic <https://github.com/yarikoptic>`__)
1852 - DOC: Datalad -> DataLad
1853 `#6937 <https://github.com/datalad/datalad/pull/6937>`__
1854 (`@aqw <https://github.com/aqw>`__)
1855 - BF: fix typo which prevented silently to not show details of
1856 filesystems `#6930 <https://github.com/datalad/datalad/pull/6930>`__
1857 (`@yarikoptic <https://github.com/yarikoptic>`__)
1858 - BF(TST): allow for a annex repo version to upgrade if running in
1859 adjusted branches
1860 `#6927 <https://github.com/datalad/datalad/pull/6927>`__
1861 (`@yarikoptic <https://github.com/yarikoptic>`__)
1862 - RF extensions github action to centralize configuration for
1863 extensions etc, use pytest for crawler
1864 `#6914 <https://github.com/datalad/datalad/pull/6914>`__
1865 (`@yarikoptic <https://github.com/yarikoptic>`__)
1866 - BF: travis - mark our directory as safe to interact with as root
1867 `#6919 <https://github.com/datalad/datalad/pull/6919>`__
1868 (`@yarikoptic <https://github.com/yarikoptic>`__)
1869 - BF: do not pretend we know what repo version git-annex would upgrade
1870 to `#6902 <https://github.com/datalad/datalad/pull/6902>`__
1871 (`@yarikoptic <https://github.com/yarikoptic>`__)
1872 - BF(TST): do not expect log message for guessing Path to be possibly a
1873 URL on windows
1874 `#6911 <https://github.com/datalad/datalad/pull/6911>`__
1875 (`@yarikoptic <https://github.com/yarikoptic>`__)
1876 - ENH(TST): Disable coverage reporting on travis while running pytest
1877 `#6898 <https://github.com/datalad/datalad/pull/6898>`__
1878 (`@yarikoptic <https://github.com/yarikoptic>`__)
1879 - RF: just rename internal variable from unclear “op” to “io”
1880 `#6907 <https://github.com/datalad/datalad/pull/6907>`__
1881 (`@yarikoptic <https://github.com/yarikoptic>`__)
1882 - DX: Demote loglevel of message on url parameters to DEBUG while
1883 guessing RI `#6891 <https://github.com/datalad/datalad/pull/6891>`__
1884 (`@adswa <https://github.com/adswa>`__
1885 `@yarikoptic <https://github.com/yarikoptic>`__)
1886 - Fix and expand datalad.runner type annotations
1887 `#6893 <https://github.com/datalad/datalad/pull/6893>`__
1888 (`@christian-monch <https://github.com/christian-monch>`__
1889 `@yarikoptic <https://github.com/yarikoptic>`__)
1890 - Use pytest to test datalad-metalad in test_extensions-workflow
1891 `#6892 <https://github.com/datalad/datalad/pull/6892>`__
1892 (`@christian-monch <https://github.com/christian-monch>`__)
1893 - Let push honor multiple publication dependencies declared via
1894 siblings `#6869 <https://github.com/datalad/datalad/pull/6869>`__
1895 (`@mih <https://github.com/mih>`__
1896 `@yarikoptic <https://github.com/yarikoptic>`__)
1897 - ENH: upgrade versioneer from versioneer-0.20.dev0 to
1898 versioneer-0.23.dev0
1899 `#6888 <https://github.com/datalad/datalad/pull/6888>`__
1900 (`@yarikoptic <https://github.com/yarikoptic>`__)
1901 - ENH: introduce typing checking and GitHub workflow
1902 `#6885 <https://github.com/datalad/datalad/pull/6885>`__
1903 (`@yarikoptic <https://github.com/yarikoptic>`__)
1904 - RF,ENH(TST): future proof testing of git annex version upgrade + test
1905 annex init on all supported versions
1906 `#6880 <https://github.com/datalad/datalad/pull/6880>`__
1907 (`@yarikoptic <https://github.com/yarikoptic>`__)
1908 - ENH(TST): test against supported git annex repo version 10 + make it
1909 a full sweep over tests
1910 `#6881 <https://github.com/datalad/datalad/pull/6881>`__
1911 (`@yarikoptic <https://github.com/yarikoptic>`__)
1912 - BF: RF f-string uses in logger to %-interpolations
1913 `#6886 <https://github.com/datalad/datalad/pull/6886>`__
1914 (`@yarikoptic <https://github.com/yarikoptic>`__)
1915 - Merge branch ‘bf-sphinx-5.1.0’ into maint
1916 `#6883 <https://github.com/datalad/datalad/pull/6883>`__
1917 (`@yarikoptic <https://github.com/yarikoptic>`__)
1918 - BF(DOC): workaround for #10701 of sphinx in 5.1.0
1919 `#6883 <https://github.com/datalad/datalad/pull/6883>`__
1920 (`@yarikoptic <https://github.com/yarikoptic>`__)
1921 - Clarify confusing INFO log message from get() on dataset installation
1922 `#6871 <https://github.com/datalad/datalad/pull/6871>`__
1923 (`@mih <https://github.com/mih>`__)
1924 - Protect again failing to load a command interface from an extension
1925 `#6879 <https://github.com/datalad/datalad/pull/6879>`__
1926 (`@mih <https://github.com/mih>`__)
1927 - Support unsetting config via ``datalad -c :<name>``
1928 `#6864 <https://github.com/datalad/datalad/pull/6864>`__
1929 (`@mih <https://github.com/mih>`__)
1930 - Fix DOC string typo in the path within AnnexRepo.annexstatus, and
1931 replace with proper sphinx reference
1932 `#6858 <https://github.com/datalad/datalad/pull/6858>`__
1933 (`@christian-monch <https://github.com/christian-monch>`__)
1934 - Improved support for saving typechanges
1935 `#6793 <https://github.com/datalad/datalad/pull/6793>`__
1936 (`@mih <https://github.com/mih>`__)
1937
1938 Pushed to ``maint``
1939 -------------------
1940
1941 - BF: Remove duplicate ds key from result record
1942 (`@adswa <https://github.com/adswa>`__)
1943 - DOC: fix capitalization of service names
1944 (`@aqw <https://github.com/aqw>`__)
1945
1946 .. _tests-25:
1947
1948 Tests
1949 -----
1950
1951 - BF(TST,workaround): just xfail failing archives test on NFS
1952 `#6912 <https://github.com/datalad/datalad/pull/6912>`__
1953 (`@yarikoptic <https://github.com/yarikoptic>`__)
1954
1955 Authors: 5
1956 ----------
1957
1958 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
1959 - Alex Waite (`@aqw <https://github.com/aqw>`__)
1960 - Christian Mnch
1961 (`@christian-monch <https://github.com/christian-monch>`__)
1962 - Michael Hanke (`@mih <https://github.com/mih>`__)
1963 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
1964
1965 --------------
1966
1967 0.17.2 (Sat Jul 16 2022)
1968 ========================
1969
1970 .. _bug-fix-3:
1971
1972 Bug Fix
1973 -------
1974
1975 - BF(TST): do proceed to proper test for error being caught for recent
1976 git-annex on windows with symlinks
1977 `#6850 <https://github.com/datalad/datalad/pull/6850>`__
1978 (`@yarikoptic <https://github.com/yarikoptic>`__)
1979 - Addressing problem testing against python 3.10 on Travis (skip more
1980 annex versions)
1981 `#6842 <https://github.com/datalad/datalad/pull/6842>`__
1982 (`@yarikoptic <https://github.com/yarikoptic>`__)
1983 - XFAIL test_runner_parametrized_protocol on python3.8 when getting
1984 duplicate output
1985 `#6837 <https://github.com/datalad/datalad/pull/6837>`__
1986 (`@yarikoptic <https://github.com/yarikoptic>`__)
1987 - BF: Make create’s check for procedures work with several again
1988 `#6841 <https://github.com/datalad/datalad/pull/6841>`__
1989 (`@adswa <https://github.com/adswa>`__)
1990 - Support older pytests
1991 `#6836 <https://github.com/datalad/datalad/pull/6836>`__
1992 (`@jwodder <https://github.com/jwodder>`__)
1993
1994 .. _authors-3-2:
1995
1996 Authors: 3
1997 ----------
1998
1999 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
2000 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
2001 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
2002
2003 --------------
2004
2005 0.17.1 (Mon Jul 11 2022)
2006 ========================
2007
2008 .. _bug-fix-4:
2009
2010 Bug Fix
2011 -------
2012
2013 - DOC: minor fix - consistent DataLad (not Datalad) in docs and
2014 CHANGELOG `#6830 <https://github.com/datalad/datalad/pull/6830>`__
2015 (`@yarikoptic <https://github.com/yarikoptic>`__)
2016 - DOC: fixup/harmonize Changelog for 0.17.0 a little
2017 `#6828 <https://github.com/datalad/datalad/pull/6828>`__
2018 (`@yarikoptic <https://github.com/yarikoptic>`__)
2019 - BF: use –python-match minor option in new datalad-installer release
2020 to match outside version of Python
2021 `#6827 <https://github.com/datalad/datalad/pull/6827>`__
2022 (`@christian-monch <https://github.com/christian-monch>`__
2023 `@yarikoptic <https://github.com/yarikoptic>`__)
2024 - Do not quote paths for ssh >= 9
2025 `#6826 <https://github.com/datalad/datalad/pull/6826>`__
2026 (`@christian-monch <https://github.com/christian-monch>`__
2027 `@yarikoptic <https://github.com/yarikoptic>`__)
2028 - Suppress DeprecationWarning to allow for distutils to be used
2029 `#6819 <https://github.com/datalad/datalad/pull/6819>`__
2030 (`@yarikoptic <https://github.com/yarikoptic>`__)
2031 - RM(TST): remove testing of datalad.test which was removed from 0.17.0
2032 `#6822 <https://github.com/datalad/datalad/pull/6822>`__
2033 (`@yarikoptic <https://github.com/yarikoptic>`__)
2034 - Avoid import of nose-based tests.utils, make skip_if_no_module() and
2035 skip_if_no_network() allowed at module level
2036 `#6817 <https://github.com/datalad/datalad/pull/6817>`__
2037 (`@jwodder <https://github.com/jwodder>`__)
2038 - BF(TST): use higher level asyncio.run instead of
2039 asyncio.get_event_loop in test_inside_async
2040 `#6808 <https://github.com/datalad/datalad/pull/6808>`__
2041 (`@yarikoptic <https://github.com/yarikoptic>`__)
2042
2043 .. _authors-3-3:
2044
2045 Authors: 3
2046 ----------
2047
2048 - Christian Mnch
2049 (`@christian-monch <https://github.com/christian-monch>`__)
2050 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
2051 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
2052
2053 --------------
2054
2055 0.17.0 (Thu Jul 7 2022) – pytest migration
2056 ==========================================
2057
2058 .. _enhancements-and-new-features-7:
2059
2060 Enhancements and new features
2061 -----------------------------
2062
2063 - “log” progress bar now reports about starting a specific action as
2064 well. `#6756 <https://github.com/datalad/datalad/pull/6756>`__ (by
2065 @yarikoptic)
2066 - Documentation and behavior of traceback reporting for log messages
2067 via ``DATALAD_LOG_TRACEBACK`` was improved to yield a more compact
2068 report. The documentation for this feature has been clarified.
2069 `#6746 <https://github.com/datalad/datalad/pull/6746>`__ (by @mih)
2070 - ``datalad unlock`` gained a progress bar.
2071 `#6704 <https://github.com/datalad/datalad/pull/6704>`__ (by @adswa)
2072 - When ``create-sibling-gitlab`` is called on non-existing subdatasets
2073 or paths it now returns an impossible result instead of no feedback
2074 at all. `#6701 <https://github.com/datalad/datalad/pull/6701>`__ (by
2075 @adswa)
2076 - ``datalad wtf`` includes a report on file system types of commonly
2077 used paths. `#6664 <https://github.com/datalad/datalad/pull/6664>`__
2078 (by @adswa)
2079 - Use next generation metadata code in search, if it is available.
2080 `#6518 <https://github.com/datalad/datalad/pull/6518>`__ (by
2081 @christian-monch)
2082
2083 Deprecations and removals
2084 -------------------------
2085
2086 - Remove unused and untested log helpers ``NoProgressLog`` and
2087 ``OnlyProgressLog``.
2088 `#6747 <https://github.com/datalad/datalad/pull/6747>`__ (by @mih)
2089 - Remove unused ``sorted_files()`` helper.
2090 `#6722 <https://github.com/datalad/datalad/pull/6722>`__ (by @adswa)
2091 - Discontinued the value ``stdout`` for use with the config variable
2092 ``datalad.log.target`` as its use would inevitably break special
2093 remote implementations.
2094 `#6675 <https://github.com/datalad/datalad/pull/6675>`__ (by
2095 @bpoldrack)
2096 - ``AnnexRepo.add_urls()`` is deprecated in favor of
2097 ``AnnexRepo.add_url_to_file()`` or a direct call to
2098 ``AnnexRepo.call_annex()``.
2099 `#6667 <https://github.com/datalad/datalad/pull/6667>`__ (by @mih)
2100 - ``datalad test`` command and supporting functionality (e.g.,
2101 ``datalad.test``) were removed.
2102 `#6273 <https://github.com/datalad/datalad/pull/6273>`__ (by
2103 @jwodder)
2104
2105 .. _bug-fixes-23:
2106
2107 Bug Fixes
2108 ---------
2109
2110 - ``export-archive`` does not rely on ``normalize_path()`` methods
2111 anymore and became more robust when called from subdirectories.
2112 `#6745 <https://github.com/datalad/datalad/pull/6745>`__ (by @adswa)
2113 - Sanitize keys before checking content availability to ensure that the
2114 content availability of files with URL- or custom backend keys is
2115 correctly determined and marked.
2116 `#6663 <https://github.com/datalad/datalad/pull/6663>`__ (by @adswa)
2117 - Ensure saving a new subdataset to a superdataset yields a valid
2118 ``.gitmodules`` record regardless of whether and how a path
2119 constraint is given to the ``save()`` call. Fixes #6547
2120 `#6790 <https://github.com/datalad/datalad/pull/6790>`__ (by @mih)
2121 - ``save`` now repairs annex symlinks broken by a ``git-mv`` operation
2122 prior recording a new dataset state. Fixes #4967
2123 `#6795 <https://github.com/datalad/datalad/pull/6795>`__ (by @mih)
2124
2125 .. _documentation-14:
2126
2127 Documentation
2128 -------------
2129
2130 - API documentation for log helpers, like ``log_progress()`` is now
2131 included in the renderer documentation.
2132 `#6746 <https://github.com/datalad/datalad/pull/6746>`__ (by @mih)
2133 - New design document on progress reporting.
2134 `#6734 <https://github.com/datalad/datalad/pull/6734>`__ (by @mih)
2135 - Explain downstream consequences of using ``--fast`` option in
2136 ``addurls``. `#6684 <https://github.com/datalad/datalad/pull/6684>`__
2137 (by @jdkent)
2138
2139 .. _internal-19:
2140
2141 Internal
2142 --------
2143
2144 - Inline code of ``create-sibling-ria`` has been refactored to an
2145 internal helper to check for siblings with particular names across
2146 dataset hierarchies in ``datalad-next``, and is reintroduced into
2147 core to modularize the code base further.
2148 `#6706 <https://github.com/datalad/datalad/pull/6706>`__ (by @adswa)
2149 - ``get_initialized_logger`` now lets a given ``logtarget`` take
2150 precedence over ``datalad.log.target``.
2151 `#6675 <https://github.com/datalad/datalad/pull/6675>`__ (by
2152 @bpoldrack)
2153 - Many uses of deprecated call options were replaced with the
2154 recommended ones.
2155 `#6273 <https://github.com/datalad/datalad/pull/6273>`__ (by
2156 @jwodder)
2157 - Get rid of ``asyncio`` import by defining few noops methods from
2158 ``asyncio.protocols.SubprocessProtocol`` directly in
2159 ``WitlessProtocol``.
2160 `#6648 <https://github.com/datalad/datalad/pull/6648>`__ (by
2161 @yarikoptic)
2162 - Consolidate ``GitRepo.remove()`` and ``AnnexRepo.remove()`` into a
2163 single implementation.
2164 `#6783 <https://github.com/datalad/datalad/pull/6783>`__ (by @mih) ##
2165 Tests
2166 - Discontinue use of ``with_testrepos`` decorator other than for the
2167 deprecation cycle for ``nose``.
2168 `#6690 <https://github.com/datalad/datalad/pull/6690>`__ (by @mih
2169 @bpoldrack) See
2170 `#6144 <https://github.com/datalad/datalad/issues/6144>`__ for full
2171 list of changes.
2172 - Remove usage of deprecated ``AnnexRepo.add_urls`` in tests.
2173 `#6683 <https://github.com/datalad/datalad/pull/6683>`__ (by
2174 @bpoldrack)
2175 - Minimalistic (adapters, no assert changes, etc) migration from
2176 ``nose`` to ``pytest``. Support functionality possibly used by
2177 extensions and relying on ``nose`` helpers is left in place to avoid
2178 affecting their run time and defer migration of their test setups..
2179 `#6273 <https://github.com/datalad/datalad/pull/6273>`__ (by
2180 @jwodder)
2181
2182 Authors: 7
2183 ----------
2184
2185 - Yaroslav Halchenko (@yarikoptic)
2186 - Michael Hanke (@mih)
2187 - Benjamin Poldrack (@bpoldrack)
2188 - Adina Wagner (@adswa)
2189 - John T. Wodder (@jwodder)
2190 - Christian Mnch (@christian-monch)
2191 - James Kent (@jdkent)
2192
2193 0.16.7 (Wed Jul 06 2022)
2194 ========================
2195
2196 .. _bug-fix-5:
2197
2198 Bug Fix
2199 -------
2200
2201 - Fix broken annex symlink after git-mv before saving + fix a race
2202 condition in ssh copy test
2203 `#6809 <https://github.com/datalad/datalad/pull/6809>`__
2204 (`@christian-monch <https://github.com/christian-monch>`__
2205 `@mih <https://github.com/mih>`__
2206 `@yarikoptic <https://github.com/yarikoptic>`__)
2207 - Do not ignore already known status info on submodules
2208 `#6790 <https://github.com/datalad/datalad/pull/6790>`__
2209 (`@mih <https://github.com/mih>`__)
2210 - Fix “common data source” test to use a valid URL (maint-based &
2211 extended edition)
2212 `#6788 <https://github.com/datalad/datalad/pull/6788>`__
2213 (`@mih <https://github.com/mih>`__
2214 `@yarikoptic <https://github.com/yarikoptic>`__)
2215 - Upload coverage from extension tests to Codecov
2216 `#6781 <https://github.com/datalad/datalad/pull/6781>`__
2217 (`@jwodder <https://github.com/jwodder>`__)
2218 - Clean up line end handling in GitRepo
2219 `#6768 <https://github.com/datalad/datalad/pull/6768>`__
2220 (`@christian-monch <https://github.com/christian-monch>`__)
2221 - Do not skip file-URL tests on windows
2222 `#6772 <https://github.com/datalad/datalad/pull/6772>`__
2223 (`@christian-monch <https://github.com/christian-monch>`__)
2224 - Fix test errors caused by updated chardet v5 release
2225 `#6777 <https://github.com/datalad/datalad/pull/6777>`__
2226 (`@christian-monch <https://github.com/christian-monch>`__)
2227 - Preserve final trailing slash in ``call_git()`` output
2228 `#6754 <https://github.com/datalad/datalad/pull/6754>`__
2229 (`@adswa <https://github.com/adswa>`__
2230 `@yarikoptic <https://github.com/yarikoptic>`__
2231 `@christian-monch <https://github.com/christian-monch>`__)
2232
2233 .. _pushed-to-maint-1:
2234
2235 Pushed to ``maint``
2236 -------------------
2237
2238 - Make sure a subdataset is saved with a complete .gitmodules record
2239 (`@mih <https://github.com/mih>`__)
2240
2241 .. _authors-5-1:
2242
2243 Authors: 5
2244 ----------
2245
2246 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
2247 - Christian Mnch
2248 (`@christian-monch <https://github.com/christian-monch>`__)
2249 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
2250 - Michael Hanke (`@mih <https://github.com/mih>`__)
2251 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
2252
2253 --------------
2254
2255 0.16.6 (Tue Jun 14 2022)
2256 ========================
2257
2258 .. _bug-fix-6:
2259
2260 Bug Fix
2261 -------
2262
2263 - Prevent duplicated result rendering when searching in default
2264 datasets `#6765 <https://github.com/datalad/datalad/pull/6765>`__
2265 (`@christian-monch <https://github.com/christian-monch>`__)
2266 - BF(workaround): skip test_ria_postclonecfg on OSX for now
2267 (`@yarikoptic <https://github.com/yarikoptic>`__)
2268 - BF(workaround to #6759): if saving credential failed, just log error
2269 and continue `#6762 <https://github.com/datalad/datalad/pull/6762>`__
2270 (`@yarikoptic <https://github.com/yarikoptic>`__)
2271 - Prevent reentry of a runner instance
2272 `#6737 <https://github.com/datalad/datalad/pull/6737>`__
2273 (`@christian-monch <https://github.com/christian-monch>`__)
2274
2275 Authors: 2
2276 ----------
2277
2278 - Christian Mnch
2279 (`@christian-monch <https://github.com/christian-monch>`__)
2280 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
2281
2282 --------------
2283
2284 0.16.5 (Wed Jun 08 2022)
2285 ========================
2286
2287 .. _bug-fix-7:
2288
2289 Bug Fix
2290 -------
2291
2292 - BF: push to github - remove datalad-push-default-first config only in
2293 non-dry run to ensure we push default branch separately in next step
2294 `#6750 <https://github.com/datalad/datalad/pull/6750>`__
2295 (`@yarikoptic <https://github.com/yarikoptic>`__)
2296 - In addition to default (system) ssh version, report configured ssh;
2297 fix ssh version parsing on Windows
2298 `#6729 <https://github.com/datalad/datalad/pull/6729>`__
2299 (`@yarikoptic <https://github.com/yarikoptic>`__)
2300
2301 Authors: 1
2302 ----------
2303
2304 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
2305
2306 --------------
2307
2308 0.16.4 (Thu Jun 02 2022)
2309 ========================
2310
2311 .. _bug-fix-8:
2312
2313 Bug Fix
2314 -------
2315
2316 - BF(TST): RO operations - add test directory into git safe.directory
2317 `#6726 <https://github.com/datalad/datalad/pull/6726>`__
2318 (`@yarikoptic <https://github.com/yarikoptic>`__)
2319 - DOC: fixup of docstring for skip_ssh
2320 `#6727 <https://github.com/datalad/datalad/pull/6727>`__
2321 (`@yarikoptic <https://github.com/yarikoptic>`__)
2322 - DOC: Set language in Sphinx config to en
2323 `#6727 <https://github.com/datalad/datalad/pull/6727>`__
2324 (`@adswa <https://github.com/adswa>`__)
2325 - BF: Catch KeyErrors from unavailable WTF infos
2326 `#6712 <https://github.com/datalad/datalad/pull/6712>`__
2327 (`@adswa <https://github.com/adswa>`__)
2328 - Add annex.private to ephemeral clones. That would make git-annex not
2329 assign shared (in git-annex branch) annex uuid.
2330 `#6702 <https://github.com/datalad/datalad/pull/6702>`__
2331 (`@bpoldrack <https://github.com/bpoldrack>`__
2332 `@adswa <https://github.com/adswa>`__)
2333 - BF: require argcomplete version at least 1.12.3 to test/operate
2334 correctly `#6693 <https://github.com/datalad/datalad/pull/6693>`__
2335 (`@yarikoptic <https://github.com/yarikoptic>`__)
2336 - Replace Zenodo DOI with JOSS for due credit
2337 `#6725 <https://github.com/datalad/datalad/pull/6725>`__
2338 (`@adswa <https://github.com/adswa>`__)
2339
2340 .. _authors-3-4:
2341
2342 Authors: 3
2343 ----------
2344
2345 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
2346 - Benjamin Poldrack (`@bpoldrack <https://github.com/bpoldrack>`__)
2347 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
2348
2349 --------------
2350
2351 0.16.3 (Thu May 12 2022)
2352 ========================
2353
2354 .. _bug-fix-9:
2355
2356 Bug Fix
2357 -------
2358
2359 - No change for a PR to trigger release
2360 `#6692 <https://github.com/datalad/datalad/pull/6692>`__
2361 (`@yarikoptic <https://github.com/yarikoptic>`__)
2362 - Sanitize keys before checking content availability to ensure correct
2363 value for keys with URL or custom backend
2364 `#6665 <https://github.com/datalad/datalad/pull/6665>`__
2365 (`@adswa <https://github.com/adswa>`__
2366 `@yarikoptic <https://github.com/yarikoptic>`__)
2367 - Change a key-value pair in drop result record
2368 `#6625 <https://github.com/datalad/datalad/pull/6625>`__
2369 (`@mslw <https://github.com/mslw>`__)
2370 - Link docs of datalad-next
2371 `#6677 <https://github.com/datalad/datalad/pull/6677>`__
2372 (`@mih <https://github.com/mih>`__)
2373 - Fix ``GitRepo.get_branch_commits_()`` to handle branch names
2374 conflicts with paths
2375 `#6661 <https://github.com/datalad/datalad/pull/6661>`__
2376 (`@mih <https://github.com/mih>`__)
2377 - OPT: AnnexJsonProtocol - avoid dragging possibly long data around
2378 `#6660 <https://github.com/datalad/datalad/pull/6660>`__
2379 (`@yarikoptic <https://github.com/yarikoptic>`__)
2380 - Remove two too prominent create() INFO log message that duplicate
2381 DEBUG log and harmonize some other log messages
2382 `#6638 <https://github.com/datalad/datalad/pull/6638>`__
2383 (`@mih <https://github.com/mih>`__
2384 `@yarikoptic <https://github.com/yarikoptic>`__)
2385 - Remove unsupported parameter create_sibling_ria(existing=None)
2386 `#6637 <https://github.com/datalad/datalad/pull/6637>`__
2387 (`@mih <https://github.com/mih>`__)
2388 - Add released plugin to .autorc to annotate PRs on when released
2389 `#6639 <https://github.com/datalad/datalad/pull/6639>`__
2390 (`@yarikoptic <https://github.com/yarikoptic>`__)
2391
2392 Authors: 4
2393 ----------
2394
2395 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
2396 - Michael Hanke (`@mih <https://github.com/mih>`__)
2397 - Micha Szczepanik (`@mslw <https://github.com/mslw>`__)
2398 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
2399
2400 --------------
2401
2402 0.16.2 (Thu Apr 21 2022)
2403 ========================
2404
2405 .. _bug-fix-10:
2406
2407 Bug Fix
2408 -------
2409
2410 - Demote (to level 1 from DEBUG) and speed-up API doc logging
2411 (parseParameters)
2412 `#6635 <https://github.com/datalad/datalad/pull/6635>`__
2413 (`@mih <https://github.com/mih>`__)
2414 - Factor out actual data transfer in push
2415 `#6618 <https://github.com/datalad/datalad/pull/6618>`__
2416 (`@christian-monch <https://github.com/christian-monch>`__)
2417 - ENH: include version of datalad in tests teardown Versions: report
2418 `#6628 <https://github.com/datalad/datalad/pull/6628>`__
2419 (`@yarikoptic <https://github.com/yarikoptic>`__)
2420 - MNT: Require importlib-metadata >=3.6 for Python < 3.10 for
2421 entry_points taking kwargs
2422 `#6631 <https://github.com/datalad/datalad/pull/6631>`__
2423 (`@effigies <https://github.com/effigies>`__)
2424 - Factor out credential handling of create-sibling-ghlike
2425 `#6627 <https://github.com/datalad/datalad/pull/6627>`__
2426 (`@mih <https://github.com/mih>`__)
2427 - BF: Fix wrong key name of annex’ JSON records
2428 `#6624 <https://github.com/datalad/datalad/pull/6624>`__
2429 (`@bpoldrack <https://github.com/bpoldrack>`__)
2430
2431 .. _pushed-to-maint-2:
2432
2433 Pushed to ``maint``
2434 -------------------
2435
2436 - Fix typo in changelog (`@mih <https://github.com/mih>`__)
2437 - [ci skip] minor typo fix
2438 (`@yarikoptic <https://github.com/yarikoptic>`__)
2439
2440 .. _authors-5-2:
2441
2442 Authors: 5
2443 ----------
2444
2445 - Benjamin Poldrack (`@bpoldrack <https://github.com/bpoldrack>`__)
2446 - Chris Markiewicz (`@effigies <https://github.com/effigies>`__)
2447 - Christian Mnch
2448 (`@christian-monch <https://github.com/christian-monch>`__)
2449 - Michael Hanke (`@mih <https://github.com/mih>`__)
2450 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
2451
2452 --------------
2453
2454 0.16.1 (Fr Apr 8 2022) – April Fools’ Release
2455 =============================================
2456
2457 - Fixes forgotten changelog in docs
2458
2459 0.16.0 (Fr Apr 8 2022) – Spring cleaning!
2460 =========================================
2461
2462 .. _enhancements-and-new-features-8:
2463
2464 Enhancements and new features
2465 -----------------------------
2466
2467 - A new set of ``create-sibling-*`` commands reimplements the
2468 GitHub-platform support of ``create-sibling-github`` and adds support
2469 to interface three new platforms in a unified fashion: GIN
2470 (``create-sibling-gin``), GOGS (``create-sibling-gogs``), and Gitea
2471 (``create-sibling-gitea``). All commands rely on personal access
2472 tokens only for authentication, allow for specifying one of several
2473 stored credentials via a uniform ``--credential`` parameter, and
2474 support a uniform ``--dry-run`` mode for testing without network.
2475 `#5949 <https://github.com/datalad/datalad/pull/5949>`__ (by @mih)
2476 - ``create-sibling-github`` now has supports direct specification of
2477 organization repositories via a ``[<org>/]repo``\ syntax
2478 `#5949 <https://github.com/datalad/datalad/pull/5949>`__ (by @mih)
2479 - ``create-sibling-gitlab`` gained a ``--dry-run`` parameter to match
2480 the corresponding parameters in
2481 ``create-sibling-{github,gin,gogs,gitea}``
2482 `#6013 <https://github.com/datalad/datalad/pull/6013>`__ (by @adswa)
2483 - The ``--new-store-ok`` parameter of ``create-sibling-ria`` only
2484 creates new RIA stores when explicitly provided
2485 `#6045 <https://github.com/datalad/datalad/pull/6045>`__ (by @adswa)
2486 - The default performance of ``status()`` and ``diff()`` commands is
2487 improved by up to 700% removing file-type evaluation as a default
2488 operation, and simplifying the type reporting rule
2489 `#6097 <https://github.com/datalad/datalad/pull/6097>`__ (by @mih)
2490 - ``drop()`` and ``remove()`` were reimplemented in full,
2491 conceptualized as the antagonist commands to ``get()`` and
2492 ``clone()``. A new, harmonized set of parameters
2493 (``--what ['filecontent', 'allkeys', 'datasets', 'all']``,
2494 ``--reckless ['modification', 'availability', 'undead', 'kill']``)
2495 simplifies their API. Both commands include additional safeguards.
2496 ``uninstall`` is replaced with a thin shim command around ``drop()``
2497 `#6111 <https://github.com/datalad/datalad/pull/6111>`__ (by @mih)
2498 - ``add_archive_content()`` was refactored into a dataset method and
2499 gained progress bars
2500 `#6105 <https://github.com/datalad/datalad/pull/6105>`__ (by @adswa)
2501 - The ``datalad`` and ``datalad-archives`` special remotes have been
2502 reimplemented based on ``AnnexRemote``
2503 `#6165 <https://github.com/datalad/datalad/pull/6165>`__ (by @mih)
2504 - The ``result_renderer()`` semantics were decomplexified and
2505 harmonized. The previous ``default`` result renderer was renamed to
2506 ``generic``. `#6174 <https://github.com/datalad/datalad/pull/6174>`__
2507 (by @mih)
2508 - ``get_status_dict`` learned to include exit codes in the case of
2509 CommandErrors
2510 `#5642 <https://github.com/datalad/datalad/pull/5642>`__ (by
2511 @yarikoptic)
2512 - ``datalad clone`` can now pass options to ``git-clone``, adding
2513 support for cloning specific tags or branches, naming siblings other
2514 names than ``origin``, and exposing ``git clone``\ ’s optimization
2515 arguments `#6218 <https://github.com/datalad/datalad/pull/6218>`__
2516 (by @kyleam and @mih)
2517 - Inactive BatchedCommands are cleaned up
2518 `#6206 <https://github.com/datalad/datalad/pull/6206>`__ (by
2519 @jwodder)
2520 - ``export-archive-ora`` learned to filter files exported to 7z
2521 archives `#6234 <https://github.com/datalad/datalad/pull/6234>`__ (by
2522 @mih and @bpinsard)
2523 - ``datalad run`` learned to glob recursively
2524 `#6262 <https://github.com/datalad/datalad/pull/6262>`__ (by @AKSoo)
2525 - The ORA remote learned to recover from interrupted uploads
2526 `#6267 <https://github.com/datalad/datalad/pull/6267>`__ (by @mih)
2527 - A new threaded runner with support for timeouts and generator-based
2528 subprocess communication is introduced and used in ``BatchedCommand``
2529 and ``AnnexRepo``
2530 `#6244 <https://github.com/datalad/datalad/pull/6244>`__ (by
2531 @christian-monch)
2532 - A new switch allows to enable librarymode and queries for the
2533 effective API in use
2534 `#6213 <https://github.com/datalad/datalad/pull/6213>`__ (by @mih)
2535 - ``run`` and ``rerun`` now support parallel jobs via ``--jobs``
2536 `#6279 <https://github.com/datalad/datalad/pull/6279>`__ (by @AKSoo)
2537 - A new ``foreach-dataset`` plumbing command allows to run commands on
2538 each (sub)dataset, similar to ``git submodule foreach``
2539 `#5517 <https://github.com/datalad/datalad/pull/5517>`__ (by
2540 @yarikoptic)
2541 - The ``dataset`` parameter is not restricted to only locally
2542 resolvable file-URLs anymore
2543 `#6276 <https://github.com/datalad/datalad/pull/6276>`__ (by
2544 @christian-monch)
2545 - DataLad’s credential system is now able to query ``git-credential``
2546 by specifying credential type ``git`` in the respective provider
2547 configuration
2548 `#5796 <https://github.com/datalad/datalad/pull/5796>`__ (by
2549 @bpoldrack)
2550 - DataLad now comes with a git credential helper
2551 ``git-credential-datalad`` allowing Git to query DataLad’s credential
2552 system `#5796 <https://github.com/datalad/datalad/pull/5796>`__ (by
2553 @bpoldrack and @mih)
2554 - The new runner now allows for multiple threads
2555 `#6371 <https://github.com/datalad/datalad/pull/6371>`__ (by
2556 @christian-monch)
2557 - A new configurationcommand provides an interface to manipulate and
2558 query the DataLad configuration.
2559 `#6306 <https://github.com/datalad/datalad/pull/6306>`__ (by @mih)
2560
2561 - Unlike the global Python-only datalad.cfg or dataset-specific
2562 Dataset.config configuration managers, this command offers a
2563 uniform API across the Python and the command line interfaces.
2564 - This command was previously available in the mihextras extension
2565 as x-configuration, and has been merged into the core package in
2566 an improved version.
2567 `#5489 <https://github.com/datalad/datalad/pull/5489>`__ (by @mih)
2568 - In its default dump mode, the command provides an annotated list
2569 of the effective configuration after considering all configuration
2570 sources, including hints on additional configuration settings and
2571 their supported values.
2572
2573 - The command line interface help-reporting has been sped up by ~20%
2574 `#6370 <https://github.com/datalad/datalad/pull/6370>`__
2575 `#6378 <https://github.com/datalad/datalad/pull/6378>`__ (by @mih)
2576 - ``ConfigManager`` now supports reading committed dataset
2577 configuration in bare repositories. Analog to reading
2578 ``.datalad/config`` from a worktree, ``blob:HEAD:.datalad/config`` is
2579 read (e.g., the config committed in the default branch). The support
2580 includes \`\ ``reload()`` change detection using the gitsha of this
2581 file. The behavior for non-bare repositories is unchanged.
2582 `#6332 <https://github.com/datalad/datalad/pull/6332>`__ (by @mih)
2583 - The CLI help generation has been sped up, and now also supports the
2584 completion of parameter values for a fixed set of choices
2585 `#6415 <https://github.com/datalad/datalad/pull/6415>`__ (by @mih)
2586 - Individual command implementations can now declare a specific
2587 “on-failure” behavior by defining ``Interface.on_failure`` to be one
2588 of the supported modes (stop, continue, ignore). Previously, such a
2589 modification was only possible on a per-call basis.
2590 `#6430 <https://github.com/datalad/datalad/pull/6430>`__ (by @mih)
2591 - The ``run`` command changed its default “on-failure” behavior from
2592 ``continue`` to ``stop``. This change prevents the execution of a
2593 command in case a declared input can not be obtained. Previously,
2594 only an error result was yielded (and run eventually yielded a
2595 non-zero exit code or an ``IncompleteResultsException``), but the
2596 execution proceeded and potentially saved a dataset modification
2597 despite incomplete inputs, in case the command succeeded. This
2598 previous default behavior can still be achieved by calling run with
2599 the equivalent of ``--on-failure continue``
2600 `#6430 <https://github.com/datalad/datalad/pull/6430>`__ (by @mih)
2601 - The \`\ ``run`` command now provides readily executable, API-specific
2602 instructions how to save the results of a command execution that
2603 failed expectedly
2604 `#6434 <https://github.com/datalad/datalad/pull/6434>`__ (by @mih)
2605 - ``create-sibling --since=^`` mode will now be as fast as
2606 ``push --since=^`` to figure out for which subdatasets to create
2607 siblings `#6436 <https://github.com/datalad/datalad/pull/6436>`__ (by
2608 @yarikoptic)
2609 - When file names contain illegal characters or reserved file names
2610 that are incompatible with Windows systems a configurable check for
2611 ``save`` (``datalad.save.windows-compat-warning``) will either do
2612 nothing (``none``), emit an incompatibility warning (``warning``,
2613 default), or cause ``save`` to error (``error``)
2614 `#6291 <https://github.com/datalad/datalad/pull/6291>`__ (by @adswa)
2615 - Improve responsiveness of ``datalad drop`` in datasets with a large
2616 annex. `#6580 <https://github.com/datalad/datalad/pull/6580>`__ (by
2617 @christian-monch)
2618 - ``save`` code might operate faster on heavy file trees
2619 `#6581 <https://github.com/datalad/datalad/pull/6581>`__ (by
2620 @yarikoptic)
2621 - Removed a per-file overhead cost for ORA when downloading over HTTP
2622 `#6609 <https://github.com/datalad/datalad/pull/6609>`__ (by
2623 @bpoldrack)
2624 - A new module ``datalad.support.extensions`` offers the utility
2625 functions ``register_config()`` and ``has_config()`` that allow
2626 extension developers to announce additional configuration items to
2627 the central configuration management.
2628 `#6601 <https://github.com/datalad/datalad/pull/6601>`__ (by @mih)
2629 - When operating in a dirty dataset, ``export-to-figshare`` now yields
2630 and impossible result instead of raising a RunTimeError
2631 `#6543 <https://github.com/datalad/datalad/pull/6543>`__ (by @adswa)
2632 - Loading DataLad extension packages has been sped-up leading to
2633 between 2x and 4x faster run times for loading individual extensions
2634 and reporting help output across all installed extensions.
2635 `#6591 <https://github.com/datalad/datalad/pull/6591>`__ (by @mih)
2636 - Introduces the configuration key ``datalad.ssh.executable``. This key
2637 allows specifying an ssh-client executable that should be used by
2638 datalad to establish ssh-connections. The default value is ``ssh``
2639 unless on a Windows system where ``$WINDIR\System32\OpenSSH\ssh.exe``
2640 exists. In this case, the value defaults to
2641 ``$WINDIR\System32\OpenSSH\ssh.exe``.
2642 `#6553 <https://github.com/datalad/datalad/pull/6553>`__ (by
2643 @christian-monch)
2644 - create-sibling should perform much faster in case of ``--since``
2645 specification since would consider only submodules related to the
2646 changes since that point.
2647 `#6528 <https://github.com/datalad/datalad/pull/6528>`__ (by
2648 @yarikoptic)
2649 - A new configuration setting
2650 ``datalad.ssh.try-use-annex-bundled-git=yes|no`` can be used to
2651 influence the default remote git-annex bundle sensing for SSH
2652 connections. This was previously done unconditionally for any call to
2653 ``datalad sshrun`` (which is also used for any SSH-related Git or
2654 git-annex functionality triggered by DataLad-internal processing) and
2655 could incur a substantial per-call runtime cost. The new default is
2656 to not perform this sensing, because for, e.g., use as
2657 GIT_SSH_COMMAND there is no expectation to have a remote git-annex
2658 installation, and even with an existing git-annex/Git bundle on the
2659 remote, it is not certain that the bundled Git version is to be
2660 preferred over any other Git installation in a user’s PATH.
2661 `#6533 <https://github.com/datalad/datalad/pull/6533>`__ (by @mih)
2662 - ``run`` now yields a result record immediately after executing a
2663 command. This allows callers to use the standard
2664 ``--on-failure switch`` to control whether dataset modifications will
2665 be saved for a command that exited with an error.
2666 `#6447 <https://github.com/datalad/datalad/pull/6447>`__ (by @mih)
2667
2668 .. _deprecations-and-removals-1:
2669
2670 Deprecations and removals
2671 -------------------------
2672
2673 - The ``--pbs-runner`` commandline option (deprecated in ``0.15.0``)
2674 was removed `#5981 <https://github.com/datalad/datalad/pull/5981>`__
2675 (by @mih)
2676 - The dependency to PyGithub was dropped
2677 `#5949 <https://github.com/datalad/datalad/pull/5949>`__ (by @mih)
2678 - ``create-sibling-github``\ ’s credential handling was trimmed down to
2679 only allow personal access tokens, because GitHub discontinued
2680 user/password based authentication
2681 `#5949 <https://github.com/datalad/datalad/pull/5949>`__ (by @mih)
2682 - ``create-sibling-gitlab``\ ’s ``--dryrun`` parameter is deprecated in
2683 favor or ``--dry-run``
2684 `#6013 <https://github.com/datalad/datalad/pull/6013>`__ (by @adswa)
2685 - Internal obsolete ``Gitrepo.*_submodule`` methods were moved to
2686 ``datalad-deprecated``
2687 `#6010 <https://github.com/datalad/datalad/pull/6010>`__ (by @mih)
2688 - ``datalad/support/versions.py`` is unused in DataLad core and removed
2689 `#6115 <https://github.com/datalad/datalad/pull/6115>`__ (by
2690 @yarikoptic)
2691 - Support for the undocumented ``datalad.api.result-renderer`` config
2692 setting has been dropped
2693 `#6174 <https://github.com/datalad/datalad/pull/6174>`__ (by @mih)
2694 - Undocumented use of ``result_renderer=None`` is replaced with
2695 ``result_renderer='disabled'``
2696 `#6174 <https://github.com/datalad/datalad/pull/6174>`__ (by @mih)
2697 - ``remove``\ ’s ``--recursive`` argument has been deprecated
2698 `#6257 <https://github.com/datalad/datalad/pull/6257>`__ (by @mih)
2699 - The use of the internal helper ``get_repo_instance()`` is
2700 discontinued and deprecated
2701 `#6268 <https://github.com/datalad/datalad/pull/6268>`__ (by @mih)
2702 - Support for Python 3.6 has been dropped
2703 (`#6286 <https://github.com/datalad/datalad/pull/6286>`__ (by
2704 @christian-monch) and
2705 `#6364 <https://github.com/datalad/datalad/pull/6364>`__ (by
2706 @yarikoptic))
2707 - All but one Singularity recipe flavor have been removed due to their
2708 limited value with the end of life of Singularity Hub
2709 `#6303 <https://github.com/datalad/datalad/pull/6303>`__ (by @mih)
2710 - All code in module datalad.cmdline was (re)moved, only
2711 datalad.cmdline.helpers.get_repo_instanceis kept for a deprecation
2712 period (by @mih)
2713 - ``datalad.interface.common_opts.eval_default`` has been deprecated.
2714 All (command-specific) defaults for common interface parameters can
2715 be read from ``Interface`` class attributes
2716 (`#6391 <https://github.com/datalad/datalad/pull/6391>`__ (by @mih)
2717 - Remove unused and untested ``datalad.interface.utils`` helpers
2718 ``cls2cmdlinename`` and ``path_is_under``
2719 `#6392 <https://github.com/datalad/datalad/pull/6392>`__ (by @mih)
2720 - An unused code path for result rendering was removed from the CLI
2721 ``main()`` `#6394 <https://github.com/datalad/datalad/pull/6394>`__
2722 (by @mih)
2723 - ``create-sibling`` will require now ``"^"`` instead of an empty
2724 string for since option
2725 `#6436 <https://github.com/datalad/datalad/pull/6436>`__ (by
2726 @yarikoptic)
2727 - ``run`` no longer raises a ``CommandError`` exception for failed
2728 commands, but yields an ``error`` result that includes a superset of
2729 the information provided by the exception. This change impacts
2730 command line usage insofar as the exit code of the underlying command
2731 is no longer relayed as the exit code of the ``run`` command call –
2732 although ``run`` continues to exit with a non-zero exit code in case
2733 of an error. For Python API users, the nature of the raised exception
2734 changes from ``CommandError`` to ``IncompleteResultsError``, and the
2735 exception handling is now configurable using the standard
2736 ``on_failure`` command argument. The original ``CommandError``
2737 exception remains available via the ``exception`` property of the
2738 newly introduced result record for the command execution, and this
2739 result record is available via ``IncompleteResultsError.failed``, if
2740 such an exception is raised.
2741 `#6447 <https://github.com/datalad/datalad/pull/6447>`__ (by @mih)
2742 - Custom cast helpers were removed from datalad core and migrated to a
2743 standalone repository https://github.com/datalad/screencaster
2744 `#6516 <https://github.com/datalad/datalad/pull/6516>`__ (by @adswa)
2745 - The ``bundled`` parameter of ``get_connection_hash()`` is now ignored
2746 and will be removed with a future release.
2747 `#6532 <https://github.com/datalad/datalad/pull/6532>`__ (by @mih)
2748 - ``BaseDownloader.fetch()`` is logging download attempts on DEBUG
2749 (previously INFO) level to avoid polluting output of higher-level
2750 commands. `#6564 <https://github.com/datalad/datalad/pull/6564>`__
2751 (by @mih)
2752
2753 .. _bug-fixes-24:
2754
2755 Bug Fixes
2756 ---------
2757
2758 - ``create-sibling-gitlab`` erroneously overwrote existing sibling
2759 configurations. A safeguard will now prevent overwriting and exit
2760 with an error result
2761 `#6015 <https://github.com/datalad/datalad/pull/6015>`__ (by @adswa)
2762 - ``create-sibling-gogs`` now relays HTTP500 errors, such as “no space
2763 left on device”
2764 `#6019 <https://github.com/datalad/datalad/pull/6019>`__ (by @mih)
2765 - ``annotate_paths()`` is removed from the last parts of code base that
2766 still contained it
2767 `#6128 <https://github.com/datalad/datalad/pull/6128>`__ (by @mih)
2768 - ``add_archive_content()`` doesn’t crash with ``--key`` and
2769 ``--use-current-dir`` anymore
2770 `#6105 <https://github.com/datalad/datalad/pull/6105>`__ (by @adswa)
2771 - ``run-procedure`` now returns an error result when a non-existent
2772 procedure name is specified
2773 `#6143 <https://github.com/datalad/datalad/pull/6143>`__ (by @mslw)
2774 - A fix for a silent failure of ``download-url --archive`` when
2775 extracting the archive
2776 `#6172 <https://github.com/datalad/datalad/pull/6172>`__ (by @adswa)
2777 - Uninitialized AnnexRepos can now be dropped
2778 `#6183 <https://github.com/datalad/datalad/pull/6183>`__ (by @mih)
2779 - Instead of raising an error, the formatters tests are skipped when
2780 the ``formatters`` module is not found
2781 `#6212 <https://github.com/datalad/datalad/pull/6212>`__ (by @adswa)
2782 - ``create-sibling-gin`` does not disable git-annex availability on Gin
2783 remotes anymore
2784 `#6230 <https://github.com/datalad/datalad/pull/6230>`__ (by @mih)
2785 - The ORA special remote messaging is fixed to not break the special
2786 remote protocol anymore and to better relay messages from exceptions
2787 to communicate underlying causes
2788 `#6242 <https://github.com/datalad/datalad/pull/6242>`__ (by @mih)
2789 - A ``keyring.delete()`` call was fixed to not call an uninitialized
2790 private attribute anymore
2791 `#6253 <https://github.com/datalad/datalad/pull/6253>`__ (by
2792 @bpoldrack)
2793 - An erroneous placement of result keyword arguments into a
2794 ``format()`` method instead of ``get_status_dict()`` of
2795 ``create-sibling-ria`` has been fixed
2796 `#6256 <https://github.com/datalad/datalad/pull/6256>`__ (by @adswa)
2797 - ``status``, ``run-procedure``, and ``metadata`` are no longer
2798 swallowing result-related messages in renderers
2799 `#6280 <https://github.com/datalad/datalad/pull/6280>`__ (by @mih)
2800 - ``uninstall`` now recommends the new ``--reckless`` parameter instead
2801 of the deprecated ``--nocheck`` parameter when reporting hints
2802 `#6277 <https://github.com/datalad/datalad/pull/6277>`__ (by @adswa)
2803 - ``download-url`` learned to handle Pathobjects
2804 `#6317 <https://github.com/datalad/datalad/pull/6317>`__ (by @adswa)
2805 - Restore default result rendering behavior broken by Key interface
2806 documentation
2807 `#6394 <https://github.com/datalad/datalad/pull/6394>`__ (by @mih)
2808 - Fix a broken check for file presence in the ``ConfigManager`` that
2809 could have caused a crash in rare cases when a config file is removed
2810 during the process runtime
2811 `#6332 <https://github.com/datalad/datalad/pull/6332>`__ (by @mih)
2812 \`- ``ConfigManager.get_from_source()`` now accesses the correct
2813 information when using the documented ``source='local'``, avoiding a
2814 crash `#6332 <https://github.com/datalad/datalad/pull/6332>`__ (by
2815 @mih)
2816 - ``run`` no longer let’s the internal call to ``save`` render its
2817 results unconditionally, but the parameterization f run determines
2818 the effective rendering format.
2819 `#6421 <https://github.com/datalad/datalad/pull/6421>`__ (by @mih)
2820 - Remove an unnecessary and misleading warning from the runner
2821 `#6425 <https://github.com/datalad/datalad/pull/6425>`__ (by
2822 @christian-monch)
2823 - A number of commands stopped to double-report results
2824 `#6446 <https://github.com/datalad/datalad/pull/6446>`__ (by @adswa)
2825 - ``create-sibling-ria`` no longer creates an ``annex/objects``
2826 directory in-store, when called with ``--no-storage-sibling``.
2827 `#6495 <https://github.com/datalad/datalad/pull/6495>`__ (by
2828 @bpoldrack )
2829 - Improve error message when an invalid URL is given to ``clone``.
2830 `#6500 <https://github.com/datalad/datalad/pull/6500>`__ (by @mih)
2831 - DataLad declares a minimum version dependency to ``keyring >= 20.0``
2832 to ensure that token-based authentication can be used.
2833 `#6515 <https://github.com/datalad/datalad/pull/6515>`__ (by @adswa)
2834 - ORA special remote tries to obtain permissions when dropping a key
2835 from a RIA store rather than just failing. Thus having the same
2836 permissions in the store’s object trees as one directly managed by
2837 git-annex would have, works just fine now.
2838 `#6493 <https://github.com/datalad/datalad/pull/6493>`__ (by
2839 @bpoldrack )
2840 - ``require_dataset()`` now uniformly raises ``NoDatasetFound`` when no
2841 dataset was found. Implementations that catch the previously
2842 documented ``InsufficientArgumentsError`` or the actually raised
2843 ``ValueError`` will continue to work, because ``NoDatasetFound`` is
2844 derived from both types.
2845 `#6521 <https://github.com/datalad/datalad/pull/6521>`__ (by @mih)
2846 - Keyboard-interactive authentication is now possibly with
2847 non-multiplexed SSH connections (i.e., when no connection sharing is
2848 possible, due to lack of socket support, for example on Windows).
2849 Previously, it was disabled forcefully by DataLad for no valid
2850 reason. `#6537 <https://github.com/datalad/datalad/pull/6537>`__ (by
2851 @mih)
2852 - Remove duplicate exception type in reporting of top-level CLI
2853 exception handler.
2854 `#6563 <https://github.com/datalad/datalad/pull/6563>`__ (by @mih)
2855 - Fixes DataLad’s parsing of git-annex’ reporting on unknown paths
2856 depending on its version and the value of the ``annex.skipunknown``
2857 config. `#6550 <https://github.com/datalad/datalad/pull/6550>`__ (by
2858 @bpoldrack)
2859 - Fix ORA special remote not properly reporting on HTTP failures.
2860 `#6535 <https://github.com/datalad/datalad/pull/6535>`__ (by
2861 @bpoldrack)
2862 - ORA special remote didn’t show per-file progress bars when
2863 downloading over HTTP
2864 `#6609 <https://github.com/datalad/datalad/pull/6609>`__ (by
2865 @bpoldrack)
2866 - ``save`` now can commit the change where file becomes a directory
2867 with a staged for commit file.
2868 `#6581 <https://github.com/datalad/datalad/pull/6581>`__ (by
2869 @yarikoptic)
2870 - ``create-sibling`` will no longer create siblings for not yet saved
2871 new subdatasets, and will now create sub-datasets nested in the
2872 subdatasets which did not yet have those siblings.
2873 `#6603 <https://github.com/datalad/datalad/pull/6603>`__ (by
2874 @yarikoptic)
2875
2876 .. _documentation-15:
2877
2878 Documentation
2879 -------------
2880
2881 - A new design document sheds light on result records
2882 `#6167 <https://github.com/datalad/datalad/pull/6167>`__ (by @mih)
2883 - The ``disabled`` result renderer mode is documented
2884 `#6174 <https://github.com/datalad/datalad/pull/6174>`__ (by @mih)
2885 - A new design document sheds light on the ``datalad`` and
2886 ``datalad-archives`` special remotes
2887 `#6181 <https://github.com/datalad/datalad/pull/6181>`__ (by @mih)
2888 - A new design document sheds light on ``BatchedCommand`` and
2889 ``BatchedAnnex``
2890 `#6203 <https://github.com/datalad/datalad/pull/6203>`__ (by
2891 @christian-monch)
2892 - A new design document sheds light on standard parameters
2893 `#6214 <https://github.com/datalad/datalad/pull/6214>`__ (by @adswa)
2894 - The DataLad project adopted the Contributor Covenant COC v2.1
2895 `#6236 <https://github.com/datalad/datalad/pull/6236>`__ (by @adswa)
2896 - Docstrings learned to include Sphinx’ “version added” and
2897 “deprecated” directives
2898 `#6249 <https://github.com/datalad/datalad/pull/6249>`__ (by @mih)
2899 - A design document sheds light on basic docstring handling and
2900 formatting `#6249 <https://github.com/datalad/datalad/pull/6249>`__
2901 (by @mih)
2902 - A new design document sheds light on position versus keyword
2903 parameter usage
2904 `#6261 <https://github.com/datalad/datalad/pull/6261>`__ (by
2905 @yarikoptic)
2906 - ``create-sibling-gin``\ ’s examples have been improved to suggest
2907 ``push`` as an additional step to ensure proper configuration
2908 `#6289 <https://github.com/datalad/datalad/pull/6289>`__ (by @mslw)
2909 - A new `document <http://docs.datalad.org/credentials.html>`__
2910 describes the credential system from a user’s perspective
2911 `#5796 <https://github.com/datalad/datalad/pull/5796>`__ (by
2912 @bpoldrack)
2913 - Enhance the `design
2914 document <http://docs.datalad.org/design/credentials.html>`__ on
2915 DataLad’s credential system
2916 `#5796 <https://github.com/datalad/datalad/pull/5796>`__ (by
2917 @bpoldrack)
2918 - The documentation of the configuration command now details all
2919 locations DataLad is reading configuration items from, and their
2920 respective rules of precedence
2921 `#6306 <https://github.com/datalad/datalad/pull/6306>`__ (by @mih)
2922 - API docs for datalad.interface.base are now included in the
2923 documentation
2924 `#6378 <https://github.com/datalad/datalad/pull/6378>`__ (by @mih)
2925 - A new design document is provided that describes the basics of the
2926 command line interface implementation
2927 `#6382 <https://github.com/datalad/datalad/pull/6382>`__ (by @mih)
2928 - The \`\ ``datalad.interface.base.Interface`` class, the basis of all
2929 DataLad command implementations, has been extensively documented to
2930 provide an overview of basic principles and customization
2931 possibilities
2932 `#6391 <https://github.com/datalad/datalad/pull/6391>`__ (by @mih)
2933 - ``--since=^`` mode of operation of ``create-sibling`` is documented
2934 now `#6436 <https://github.com/datalad/datalad/pull/6436>`__ (by
2935 @yarikoptic)
2936
2937 .. _internal-20:
2938
2939 Internal
2940 --------
2941
2942 - The internal ``status()`` helper was equipped with docstrings and
2943 promotes “breadth-first” reporting with a new parameter
2944 ``reporting_order``
2945 `#6006 <https://github.com/datalad/datalad/pull/6006>`__ (by @mih)
2946 - ``AnnexRepo.get_file_annexinfo()`` is introduced for more convenient
2947 queries for single files and replaces a now deprecated
2948 ``AnnexRepo.get_file_key()`` to receive information with fewer calls
2949 to Git `#6104 <https://github.com/datalad/datalad/pull/6104>`__ (by
2950 @mih)
2951 - A new ``get_paths_by_ds()`` helper exposes ``status``\ ’ path
2952 normalization and sorting
2953 `#6110 <https://github.com/datalad/datalad/pull/6110>`__ (by @mih)
2954 - ``status`` is optimized with a cache for dataset roots
2955 `#6137 <https://github.com/datalad/datalad/pull/6137>`__ (by
2956 @yarikoptic)
2957 - The internal ``get_func_args_doc()`` helper with Python 2 is removed
2958 from DataLad core
2959 `#6175 <https://github.com/datalad/datalad/pull/6175>`__ (by
2960 @yarikoptic)
2961 - Further restructuring of the source tree to better reflect the
2962 internal dependency structure of the code: ``AddArchiveContent`` is
2963 moved from ``datalad/interface`` to ``datalad/local``
2964 (`#6188 <https://github.com/datalad/datalad/pull/6188>`__ (by @mih)),
2965 ``Clean`` is moved from ``datalad/interface`` to ``datalad/local``
2966 (`#6191 <https://github.com/datalad/datalad/pull/6191>`__ (by @mih)),
2967 ``Unlock`` is moved from ``datalad/interface`` to ``datalad/local``
2968 (`#6192 <https://github.com/datalad/datalad/pull/6192>`__ (by @mih)),
2969 ``DownloadURL`` is moved from ``datalad/interface`` to
2970 ``datalad/local``
2971 (`#6217 <https://github.com/datalad/datalad/pull/6217>`__ (by @mih)),
2972 ``Rerun`` is moved from ``datalad/interface`` to ``datalad/local``
2973 (`#6220 <https://github.com/datalad/datalad/pull/6220>`__ (by @mih)),
2974 ``RunProcedure`` is moved from ``datalad/interface`` to
2975 ``datalad/local``
2976 (`#6222 <https://github.com/datalad/datalad/pull/6222>`__ (by @mih)).
2977 The interface command list is restructured and resorted
2978 `#6223 <https://github.com/datalad/datalad/pull/6223>`__ (by @mih)
2979 - ``wrapt`` is replaced with functools’ ``wraps``
2980 `#6190 <https://github.com/datalad/datalad/pull/6190>`__ (by
2981 @yariktopic)
2982 - The unmaintained ``appdirs`` library has been replaced with
2983 ``platformdirs``
2984 `#6198 <https://github.com/datalad/datalad/pull/6198>`__ (by @adswa)
2985 - Modelines mismatching the code style in source files were fixed
2986 `#6263 <https://github.com/datalad/datalad/pull/6263>`__ (by @AKSoo)
2987 - ``datalad/__init__.py`` has been cleaned up
2988 `#6271 <https://github.com/datalad/datalad/pull/6271>`__ (by @mih)
2989 - ``GitRepo.call_git_items`` is implemented with a generator-based
2990 runner `#6278 <https://github.com/datalad/datalad/pull/6278>`__ (by
2991 @christian-monch)
2992 - Separate positional from keyword arguments in the Python API to match
2993 CLI with ``*``
2994 `#6176 <https://github.com/datalad/datalad/pull/6176>`__ (by
2995 @yarikoptic),
2996 `#6304 <https://github.com/datalad/datalad/pull/6304>`__ (by
2997 @christian-monch)
2998 - ``GitRepo.bare`` does not require the ConfigManager anymore
2999 `#6323 <https://github.com/datalad/datalad/pull/6323>`__ (by @mih)
3000 - ``_get_dot_git()`` was reimplemented to be more efficient and
3001 consistent, by testing for common scenarios first and introducing a
3002 consistently applied ``resolved`` flag for result path reporting
3003 `#6325 <https://github.com/datalad/datalad/pull/6325>`__ (by @mih)
3004 - All data files under ``datalad`` are now included when installing
3005 DataLad `#6336 <https://github.com/datalad/datalad/pull/6336>`__ (by
3006 @jwodder)
3007 - Add internal method for non-interactive provider/credential storing
3008 `#5796 <https://github.com/datalad/datalad/pull/5796>`__ (by
3009 @bpoldrack)
3010 - Allow credential classes to have a context set, consisting of a URL
3011 they are to be used with and a dataset DataLad is operating on,
3012 allowing to consider “local” and “dataset” config locations
3013 `#5796 <https://github.com/datalad/datalad/pull/5796>`__ (by
3014 @bpoldrack)
3015 - The Interface method ``get_refds_path()`` was deprecated
3016 `#6387 <https://github.com/datalad/datalad/pull/6387>`__ (by @adswa)
3017 - ``datalad.interface.base.Interface`` is now an abstract class
3018 `#6391 <https://github.com/datalad/datalad/pull/6391>`__ (by @mih)
3019 - Simplified the decision making for result rendering, and reduced code
3020 complexity `#6394 <https://github.com/datalad/datalad/pull/6394>`__
3021 (by @mih)
3022 - Reduce code duplication in ``datalad.support.json_py``
3023 `#6398 <https://github.com/datalad/datalad/pull/6398>`__ (by @mih)
3024 - Use public ``ArgumentParser.parse_known_args`` instead of protected
3025 ``_parse_known_args``
3026 `#6414 <https://github.com/datalad/datalad/pull/6414>`__ (by
3027 @yarikoptic)
3028 - ``add-archive-content`` does not rely on the deprecated
3029 ``tempfile.mktemp`` anymore, but uses the more secure
3030 ``tempfile.mkdtemp``
3031 `#6428 <https://github.com/datalad/datalad/pull/6428>`__ (by @adswa)
3032 - AnnexRepo’s internal ``annexstatus`` is deprecated. In its place, a
3033 new test helper assists the few tests that rely on it
3034 `#6413 <https://github.com/datalad/datalad/pull/6413>`__ (by @adswa)
3035 - ``config`` has been refactored from ``where[="dataset"]`` to
3036 ``scope[="branch"]``
3037 `#5969 <https://github.com/datalad/datalad/pull/5969>`__ (by
3038 @yarikoptic)
3039 - Common command arguments are now uniformly and exhaustively passed to
3040 result renderers and filters for decision making. Previously, the
3041 presence of a particular argument depended on the respective API and
3042 circumstances of a command call.
3043 `#6440 <https://github.com/datalad/datalad/pull/6440>`__ (by @mih)
3044 - Entrypoint processing for extensions and metadata extractors has been
3045 consolidated on a uniform helper that is about twice as fast as the
3046 previous implementations.
3047 `#6591 <https://github.com/datalad/datalad/pull/6591>`__ (by @mih)
3048
3049 .. _tests-26:
3050
3051 Tests
3052 -----
3053
3054 - A range of Windows tests pass and were enabled
3055 `#6136 <https://github.com/datalad/datalad/pull/6136>`__ (by @adswa)
3056 - Invalid escape sequences in some tests were fixed
3057 `#6147 <https://github.com/datalad/datalad/pull/6147>`__ (by @mih)
3058 - A cross-platform compatible HTTP-serving test environment is
3059 introduced `#6153 <https://github.com/datalad/datalad/pull/6153>`__
3060 (by @mih)
3061 - A new helper exposes ``serve_path_via_http`` to the command line to
3062 deploy an ad-hoc instance of the HTTP server used for internal
3063 testing, with SSL and auth, if desired.
3064 `#6169 <https://github.com/datalad/datalad/pull/6169>`__ (by @mih)
3065 - Windows tests were redistributed across worker runs to harmonize
3066 runtime `#6200 <https://github.com/datalad/datalad/pull/6200>`__ (by
3067 @adswa)
3068 - ``Batchedcommand`` gained a basic test
3069 `#6203 <https://github.com/datalad/datalad/pull/6203>`__ (by
3070 @christian-monch)
3071 - The use of ``with_testrepo`` is discontinued in all core tests
3072 `#6224 <https://github.com/datalad/datalad/pull/6224>`__ (by @mih)
3073 - The new ``git-annex.filter.annex.process`` configuration is enabled
3074 by default on Windows to speed up the test suite
3075 `#6245 <https://github.com/datalad/datalad/pull/6245>`__ (by @mih)
3076 - If the available Git version supports it, the test suite now uses
3077 ``GIT_CONFIG_GLOBAL`` to configure a fake home directory instead of
3078 overwriting ``HOME`` on OSX
3079 (`#6251 <https://github.com/datalad/datalad/pull/6251>`__ (by
3080 @bpoldrack)) and ``HOME`` and ``USERPROFILE`` on Windows
3081 `#6260 <https://github.com/datalad/datalad/pull/6260>`__ (by @adswa)
3082 - Windows test timeouts of runners were addressed
3083 `#6311 <https://github.com/datalad/datalad/pull/6311>`__ (by
3084 @christian-monch)
3085 - A handful of Windows tests were fixed
3086 (`#6352 <https://github.com/datalad/datalad/pull/6352>`__ (by
3087 @yarikoptic)) or disabled
3088 (`#6353 <https://github.com/datalad/datalad/pull/6353>`__ (by
3089 @yarikoptic))
3090 - ``download-url``\ ’s test under ``http_proxy`` are skipped when a
3091 session can’t be established
3092 `#6361 <https://github.com/datalad/datalad/pull/6361>`__ (by
3093 @yarikoptic)
3094 - A test for ``datalad clean`` was fixed to be invoked within a dataset
3095 `#6359 <https://github.com/datalad/datalad/pull/6359>`__ (by
3096 @yarikoptic)
3097 - The new datalad.cli.tests have an improved module coverage of 80%
3098 `#6378 <https://github.com/datalad/datalad/pull/6378>`__ (by @mih)
3099 - The ``test_source_candidate_subdataset`` has been marked as ``@slow``
3100 `#6429 <https://github.com/datalad/datalad/pull/6429>`__ (by
3101 @yarikoptic)
3102 - Dedicated ``CLI`` benchmarks exist now
3103 `#6381 <https://github.com/datalad/datalad/pull/6381>`__ (by @mih)
3104 - Enable code coverage report for subprocesses
3105 `#6546 <https://github.com/datalad/datalad/pull/6546>`__ (by @adswa)
3106 - Skip a test on annex>=10.20220127 due to a bug in annex. See
3107 https://git-annex.branchable.com/bugs/Change_to_annex.largefiles_leaves_repo_modified/
3108
3109 Infra
3110 -----
3111
3112 - A new issue template using GitHub forms prestructures bug reports
3113 `#6048 <https://github.com/datalad/datalad/pull/6048>`__ (by
3114 @Remi-Gau)
3115 - DataLad and its dependency stack were packaged for Gentoo Linux
3116 `#6088 <https://github.com/datalad/datalad/pull/6088>`__ (by
3117 @TheChymera)
3118 - The readthedocs configuration is modernized to version 2
3119 `#6207 <https://github.com/datalad/datalad/pull/6207>`__ (by @adswa)
3120 - The Windows CI setup now runs on Appveyor’s Visual Studio 2022
3121 configuration
3122 `#6228 <https://github.com/datalad/datalad/pull/6228>`__ (by @adswa)
3123 - The ``readthedocs-theme`` and ``Sphinx`` versions were pinned to
3124 re-enable rendering of bullet points in the documentation
3125 `#6346 <https://github.com/datalad/datalad/pull/6346>`__ (by @adswa)
3126 - The PR template was updated with a CHANGELOG template. Future PRs
3127 should use it to include a summary for the CHANGELOG
3128 `#6396 <https://github.com/datalad/datalad/pull/6396>`__ (by @mih)
3129
3130 Authors: 11
3131 -----------
3132
3133 - Michael Hanke (@mih)
3134 - Yaroslav Halchenko (@yarikoptic)
3135 - Adina Wagner (@adswa)
3136 - Remi Gau (@Remi-Gau)
3137 - Horea Christian (@TheChymera)
3138 - Micha Szczepanik (@mslw)
3139 - Christian Mnch (@christian-monch)
3140 - John T. Wodder (@jwodder)
3141 - Benjamin Poldrack (@bpoldrack)
3142 - Sin Kim (@AKSoo)
3143 - Basile Pinsard (@bpinsard)
3144
3145 --------------
3146
3147 0.15.6 (Sun Feb 27 2022)
3148 ========================
3149
3150 .. _bug-fix-11:
3151
3152 Bug Fix
3153 -------
3154
3155 - BF: do not use BaseDownloader instance wide InterProcessLock -
3156 resolves stalling or errors during parallel installs
3157 `#6507 <https://github.com/datalad/datalad/pull/6507>`__
3158 (`@yarikoptic <https://github.com/yarikoptic>`__)
3159 - release workflow: add -vv to auto invocation
3160 (`@yarikoptic <https://github.com/yarikoptic>`__)
3161 - Fix version incorrectly incremented by release process in CHANGELOGs
3162 `#6459 <https://github.com/datalad/datalad/pull/6459>`__
3163 (`@yarikoptic <https://github.com/yarikoptic>`__)
3164 - BF(TST): add another condition to skip under http_proxy set
3165 `#6459 <https://github.com/datalad/datalad/pull/6459>`__
3166 (`@yarikoptic <https://github.com/yarikoptic>`__)
3167
3168 .. _authors-1-1:
3169
3170 Authors: 1
3171 ----------
3172
3173 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
3174
3175 --------------
3176
3177 0.15.5 (Wed Feb 09 2022)
3178 ========================
3179
3180 Enhancement
3181 -----------
3182
3183 - BF: When download-url gets Pathobject as path convert it to a string
3184 `#6364 <https://github.com/datalad/datalad/pull/6364>`__
3185 (`@adswa <https://github.com/adswa>`__)
3186
3187 .. _bug-fix-12:
3188
3189 Bug Fix
3190 -------
3191
3192 - Fix AnnexRepo.whereis key=True mode operation, and add batch mode
3193 support `#6379 <https://github.com/datalad/datalad/pull/6379>`__
3194 (`@yarikoptic <https://github.com/yarikoptic>`__)
3195 - DOC: run - adjust description for -i/-o to mention that it could be a
3196 directory `#6416 <https://github.com/datalad/datalad/pull/6416>`__
3197 (`@yarikoptic <https://github.com/yarikoptic>`__)
3198 - BF: ORA over HTTP tried to check archive
3199 `#6355 <https://github.com/datalad/datalad/pull/6355>`__
3200 (`@bpoldrack <https://github.com/bpoldrack>`__
3201 `@yarikoptic <https://github.com/yarikoptic>`__)
3202 - BF: condition access to isatty to have stream eval to True
3203 `#6360 <https://github.com/datalad/datalad/pull/6360>`__
3204 (`@yarikoptic <https://github.com/yarikoptic>`__)
3205 - BF: python 3.10 compatibility fixes
3206 `#6363 <https://github.com/datalad/datalad/pull/6363>`__
3207 (`@yarikoptic <https://github.com/yarikoptic>`__)
3208 - Remove two(!) copies of a test
3209 `#6374 <https://github.com/datalad/datalad/pull/6374>`__
3210 (`@mih <https://github.com/mih>`__)
3211 - Warn just once about incomplete git config
3212 `#6343 <https://github.com/datalad/datalad/pull/6343>`__
3213 (`@yarikoptic <https://github.com/yarikoptic>`__)
3214 - Make version detection robust to GIT_DIR specification
3215 `#6341 <https://github.com/datalad/datalad/pull/6341>`__
3216 (`@effigies <https://github.com/effigies>`__
3217 `@mih <https://github.com/mih>`__)
3218 - BF(Q&D): do not crash - issue warning - if template fails to format
3219 `#6319 <https://github.com/datalad/datalad/pull/6319>`__
3220 (`@yarikoptic <https://github.com/yarikoptic>`__)
3221
3222 .. _authors-5-3:
3223
3224 Authors: 5
3225 ----------
3226
3227 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
3228 - Benjamin Poldrack (`@bpoldrack <https://github.com/bpoldrack>`__)
3229 - Chris Markiewicz (`@effigies <https://github.com/effigies>`__)
3230 - Michael Hanke (`@mih <https://github.com/mih>`__)
3231 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
3232
3233 --------------
3234
3235 0.15.4 (Thu Dec 16 2021)
3236 ========================
3237
3238 .. _bug-fix-13:
3239
3240 Bug Fix
3241 -------
3242
3243 - BF: autorc - replace incorrect releaseTypes with “none”
3244 `#6320 <https://github.com/datalad/datalad/pull/6320>`__
3245 (`@yarikoptic <https://github.com/yarikoptic>`__)
3246 - Minor enhancement to CONTRIBUTING.md
3247 `#6309 <https://github.com/datalad/datalad/pull/6309>`__
3248 (`@bpoldrack <https://github.com/bpoldrack>`__)
3249 - UX: If a clean repo is dirty after a failed run, give clean-up hints
3250 `#6112 <https://github.com/datalad/datalad/pull/6112>`__
3251 (`@adswa <https://github.com/adswa>`__)
3252 - Stop using distutils
3253 `#6113 <https://github.com/datalad/datalad/pull/6113>`__
3254 (`@jwodder <https://github.com/jwodder>`__)
3255 - BF: RIARemote - set UI backend to annex to make it interactive
3256 `#6287 <https://github.com/datalad/datalad/pull/6287>`__
3257 (`@yarikoptic <https://github.com/yarikoptic>`__
3258 `@bpoldrack <https://github.com/bpoldrack>`__)
3259 - Fix invalid escape sequences
3260 `#6293 <https://github.com/datalad/datalad/pull/6293>`__
3261 (`@jwodder <https://github.com/jwodder>`__)
3262 - CI: Update environment for windows CI builds
3263 `#6292 <https://github.com/datalad/datalad/pull/6292>`__
3264 (`@bpoldrack <https://github.com/bpoldrack>`__)
3265 - bump the python version used for mac os tests
3266 `#6288 <https://github.com/datalad/datalad/pull/6288>`__
3267 (`@christian-monch <https://github.com/christian-monch>`__
3268 `@bpoldrack <https://github.com/bpoldrack>`__)
3269 - ENH(UX): log a hint to use ulimit command in case of “Too long”
3270 exception `#6173 <https://github.com/datalad/datalad/pull/6173>`__
3271 (`@yarikoptic <https://github.com/yarikoptic>`__)
3272 - Report correct HTTP URL for RIA store content
3273 `#6091 <https://github.com/datalad/datalad/pull/6091>`__
3274 (`@mih <https://github.com/mih>`__)
3275 - BF: Don’t overwrite subdataset source candidates
3276 `#6168 <https://github.com/datalad/datalad/pull/6168>`__
3277 (`@bpoldrack <https://github.com/bpoldrack>`__)
3278 - Bump sphinx requirement to bypass readthedocs defaults
3279 `#6189 <https://github.com/datalad/datalad/pull/6189>`__
3280 (`@mih <https://github.com/mih>`__)
3281 - infra: Provide custom prefix to auto-related labels
3282 `#6151 <https://github.com/datalad/datalad/pull/6151>`__
3283 (`@adswa <https://github.com/adswa>`__)
3284 - Remove all usage of exc_str()
3285 `#6142 <https://github.com/datalad/datalad/pull/6142>`__
3286 (`@mih <https://github.com/mih>`__)
3287 - BF: obtain information about annex special remotes also from annex
3288 journal `#6135 <https://github.com/datalad/datalad/pull/6135>`__
3289 (`@yarikoptic <https://github.com/yarikoptic>`__
3290 `@mih <https://github.com/mih>`__)
3291 - BF: clone tried to save new subdataset despite failing to clone
3292 `#6140 <https://github.com/datalad/datalad/pull/6140>`__
3293 (`@bpoldrack <https://github.com/bpoldrack>`__)
3294
3295 .. _tests-27:
3296
3297 Tests
3298 -----
3299
3300 - RF+BF: use skip_if_no_module helper instead of try/except for libxmp
3301 and boto `#6148 <https://github.com/datalad/datalad/pull/6148>`__
3302 (`@yarikoptic <https://github.com/yarikoptic>`__)
3303 - git://github.com -> https://github.com
3304 `#6134 <https://github.com/datalad/datalad/pull/6134>`__
3305 (`@mih <https://github.com/mih>`__)
3306
3307 Authors: 6
3308 ----------
3309
3310 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
3311 - Benjamin Poldrack (`@bpoldrack <https://github.com/bpoldrack>`__)
3312 - Christian Mnch
3313 (`@christian-monch <https://github.com/christian-monch>`__)
3314 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
3315 - Michael Hanke (`@mih <https://github.com/mih>`__)
3316 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
3317
3318 --------------
3319
3320 0.15.3 (Sat Oct 30 2021)
3321 ========================
3322
3323 .. _bug-fix-14:
3324
3325 Bug Fix
3326 -------
3327
3328 - BF: Don’t make create-sibling recursive by default
3329 `#6116 <https://github.com/datalad/datalad/pull/6116>`__
3330 (`@adswa <https://github.com/adswa>`__)
3331 - BF: Add dashes to ‘force’ option in non-empty directory error message
3332 `#6078 <https://github.com/datalad/datalad/pull/6078>`__
3333 (`@DisasterMo <https://github.com/DisasterMo>`__)
3334 - DOC: Add supported URL types to download-url’s docstring
3335 `#6098 <https://github.com/datalad/datalad/pull/6098>`__
3336 (`@adswa <https://github.com/adswa>`__)
3337 - BF: Retain git-annex error messages & don’t show them if operation
3338 successful `#6070 <https://github.com/datalad/datalad/pull/6070>`__
3339 (`@DisasterMo <https://github.com/DisasterMo>`__)
3340 - Remove uses of ``__full_version__`` and ``datalad.version``
3341 `#6073 <https://github.com/datalad/datalad/pull/6073>`__
3342 (`@jwodder <https://github.com/jwodder>`__)
3343 - BF: ORA shouldn’t crash while handling a failure
3344 `#6063 <https://github.com/datalad/datalad/pull/6063>`__
3345 (`@bpoldrack <https://github.com/bpoldrack>`__)
3346 - DOC: Refine –reckless docstring on usage and wording
3347 `#6043 <https://github.com/datalad/datalad/pull/6043>`__
3348 (`@adswa <https://github.com/adswa>`__)
3349 - BF: archives upon strip - use rmtree which retries etc instead of
3350 rmdir `#6064 <https://github.com/datalad/datalad/pull/6064>`__
3351 (`@yarikoptic <https://github.com/yarikoptic>`__)
3352 - BF: do not leave test in a tmp dir destined for removal
3353 `#6059 <https://github.com/datalad/datalad/pull/6059>`__
3354 (`@yarikoptic <https://github.com/yarikoptic>`__)
3355 - Next wave of exc_str() removals
3356 `#6022 <https://github.com/datalad/datalad/pull/6022>`__
3357 (`@mih <https://github.com/mih>`__)
3358
3359 .. _pushed-to-maint-3:
3360
3361 Pushed to ``maint``
3362 -------------------
3363
3364 - CI: Enable new codecov uploader in Appveyor CI
3365 (`@adswa <https://github.com/adswa>`__)
3366
3367 .. _internal-21:
3368
3369 Internal
3370 --------
3371
3372 - UX: Log clone-candidate number and URLs
3373 `#6092 <https://github.com/datalad/datalad/pull/6092>`__
3374 (`@adswa <https://github.com/adswa>`__)
3375 - UX/ENH: Disable reporting, and don’t do superfluous internal
3376 subdatasets calls
3377 `#6094 <https://github.com/datalad/datalad/pull/6094>`__
3378 (`@adswa <https://github.com/adswa>`__)
3379 - Update codecov action to v2
3380 `#6072 <https://github.com/datalad/datalad/pull/6072>`__
3381 (`@jwodder <https://github.com/jwodder>`__)
3382
3383 .. _documentation-16:
3384
3385 Documentation
3386 -------------
3387
3388 - Design document on URL substitution feature
3389 `#6065 <https://github.com/datalad/datalad/pull/6065>`__
3390 (`@mih <https://github.com/mih>`__)
3391
3392 .. _tests-28:
3393
3394 Tests
3395 -----
3396
3397 - BF(TST): remove reuse of the same tape across unrelated tests
3398 `#6127 <https://github.com/datalad/datalad/pull/6127>`__
3399 (`@yarikoptic <https://github.com/yarikoptic>`__)
3400 - Fail Travis tests on deprecation warnings
3401 `#6074 <https://github.com/datalad/datalad/pull/6074>`__
3402 (`@jwodder <https://github.com/jwodder>`__)
3403 - Ux get result handling broken
3404 `#6052 <https://github.com/datalad/datalad/pull/6052>`__
3405 (`@christian-monch <https://github.com/christian-monch>`__)
3406 - enable metalad tests again
3407 `#6060 <https://github.com/datalad/datalad/pull/6060>`__
3408 (`@christian-monch <https://github.com/christian-monch>`__)
3409
3410 .. _authors-7-1:
3411
3412 Authors: 7
3413 ----------
3414
3415 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
3416 - Benjamin Poldrack (`@bpoldrack <https://github.com/bpoldrack>`__)
3417 - Christian Mnch
3418 (`@christian-monch <https://github.com/christian-monch>`__)
3419 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
3420 - Michael Burgardt (`@DisasterMo <https://github.com/DisasterMo>`__)
3421 - Michael Hanke (`@mih <https://github.com/mih>`__)
3422 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
3423
3424 --------------
3425
3426 0.15.2 (Wed Oct 06 2021)
3427 ========================
3428
3429 .. _bug-fix-15:
3430
3431 Bug Fix
3432 -------
3433
3434 - BF: Don’t suppress datalad subdatasets output
3435 `#6035 <https://github.com/datalad/datalad/pull/6035>`__
3436 (`@DisasterMo <https://github.com/DisasterMo>`__
3437 `@mih <https://github.com/mih>`__)
3438 - Honor datalad.runtime.use-patool if set regardless of OS (was Windows
3439 only) `#6033 <https://github.com/datalad/datalad/pull/6033>`__
3440 (`@mih <https://github.com/mih>`__)
3441 - Discontinue usage of deprecated (public) helper
3442 `#6032 <https://github.com/datalad/datalad/pull/6032>`__
3443 (`@mih <https://github.com/mih>`__)
3444 - BF: ProgressHandler - close the other handler if was specified
3445 `#6020 <https://github.com/datalad/datalad/pull/6020>`__
3446 (`@yarikoptic <https://github.com/yarikoptic>`__)
3447 - UX: Report GitLab weburl of freshly created projects in the result
3448 `#6017 <https://github.com/datalad/datalad/pull/6017>`__
3449 (`@adswa <https://github.com/adswa>`__)
3450 - Ensure there’s a blank line between the class ``__doc__`` and
3451 “Parameters” in ``build_doc`` docstrings
3452 `#6004 <https://github.com/datalad/datalad/pull/6004>`__
3453 (`@jwodder <https://github.com/jwodder>`__)
3454 - Large code-reorganization of everything runner-related
3455 `#6008 <https://github.com/datalad/datalad/pull/6008>`__
3456 (`@mih <https://github.com/mih>`__)
3457 - Discontinue exc_str() in all modern parts of the code base
3458 `#6007 <https://github.com/datalad/datalad/pull/6007>`__
3459 (`@mih <https://github.com/mih>`__)
3460
3461 .. _tests-29:
3462
3463 Tests
3464 -----
3465
3466 - TST: Add test to ensure functionality with subdatasets starting with
3467 a hyphen (-) `#6042 <https://github.com/datalad/datalad/pull/6042>`__
3468 (`@DisasterMo <https://github.com/DisasterMo>`__)
3469 - BF(TST): filter away warning from coverage from analysis of stderr of
3470 –help `#6028 <https://github.com/datalad/datalad/pull/6028>`__
3471 (`@yarikoptic <https://github.com/yarikoptic>`__)
3472 - BF: disable outdated SSL root certificate breaking chain on
3473 older/buggy clients
3474 `#6027 <https://github.com/datalad/datalad/pull/6027>`__
3475 (`@yarikoptic <https://github.com/yarikoptic>`__)
3476 - BF: start global test_http_server only if not running already
3477 `#6023 <https://github.com/datalad/datalad/pull/6023>`__
3478 (`@yarikoptic <https://github.com/yarikoptic>`__)
3479
3480 .. _authors-5-4:
3481
3482 Authors: 5
3483 ----------
3484
3485 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
3486 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
3487 - Michael Burgardt (`@DisasterMo <https://github.com/DisasterMo>`__)
3488 - Michael Hanke (`@mih <https://github.com/mih>`__)
3489 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
3490
3491 --------------
3492
3493 0.15.1 (Fri Sep 24 2021)
3494 ========================
3495
3496 .. _bug-fix-16:
3497
3498 Bug Fix
3499 -------
3500
3501 - BF: downloader - fail to download even on non-crippled FS if symlink
3502 exists `#5991 <https://github.com/datalad/datalad/pull/5991>`__
3503 (`@yarikoptic <https://github.com/yarikoptic>`__)
3504 - ENH: import datalad.api to bind extensions methods for discovery of
3505 dataset methods
3506 `#5999 <https://github.com/datalad/datalad/pull/5999>`__
3507 (`@yarikoptic <https://github.com/yarikoptic>`__)
3508 - Restructure cmdline API presentation
3509 `#5988 <https://github.com/datalad/datalad/pull/5988>`__
3510 (`@mih <https://github.com/mih>`__)
3511 - Close file descriptors after process exit
3512 `#5983 <https://github.com/datalad/datalad/pull/5983>`__
3513 (`@mih <https://github.com/mih>`__)
3514
3515 .. _pushed-to-maint-4:
3516
3517 Pushed to ``maint``
3518 -------------------
3519
3520 - Discontinue testing of hirni extension
3521 (`@mih <https://github.com/mih>`__)
3522
3523 .. _internal-22:
3524
3525 Internal
3526 --------
3527
3528 - Add debugging information to release step
3529 `#5980 <https://github.com/datalad/datalad/pull/5980>`__
3530 (`@jwodder <https://github.com/jwodder>`__)
3531
3532 .. _documentation-17:
3533
3534 Documentation
3535 -------------
3536
3537 - Coarse description of the credential subsystem’s functionality
3538 `#5998 <https://github.com/datalad/datalad/pull/5998>`__
3539 (`@mih <https://github.com/mih>`__)
3540
3541 .. _tests-30:
3542
3543 Tests
3544 -----
3545
3546 - BF(TST): use sys.executable, mark test_ria_basics.test_url_keys as
3547 requiring network
3548 `#5986 <https://github.com/datalad/datalad/pull/5986>`__
3549 (`@yarikoptic <https://github.com/yarikoptic>`__)
3550
3551 .. _authors-3-5:
3552
3553 Authors: 3
3554 ----------
3555
3556 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
3557 - Michael Hanke (`@mih <https://github.com/mih>`__)
3558 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
3559
3560 --------------
3561
3562 0.15.0 (Tue Sep 14 2021) – We miss you Kyle!
3563 ============================================
3564
3565 .. _enhancements-and-new-features-9:
3566
3567 Enhancements and new features
3568 -----------------------------
3569
3570 - Command execution is now performed by a new ``Runner`` implementation
3571 that is no longer based on the ``asyncio`` framework, which was found
3572 to exhibit fragile performance in interaction with other
3573 ``asyncio``-using code, such as Jupyter notebooks. The new
3574 implementation is based on threads. It also supports the
3575 specification of “protocols” that were introduced with the switch to
3576 the ``asyncio`` implementation in 0.14.0.
3577 (`#5667 <https://github.com/datalad/datalad/issues/5667>`__)
3578
3579 - ``clone`` now supports arbitrary URL transformations based on regular
3580 expressions. One or more transformation steps can be defined via
3581 ``datalad.clone.url-substitute.<label>`` configuration settings. The
3582 feature can be (and is now) used to support convenience mappings,
3583 such as ``https://osf.io/q8xnk/`` (displayed in a browser window) to
3584 ``osf://q8xnk`` (clonable via the ``datalad-osf`` extension.
3585 (`#5749 <https://github.com/datalad/datalad/issues/5749>`__)
3586
3587 - Homogenize SSH use and configurability between DataLad and git-annex,
3588 by instructing git-annex to use DataLad’s ``sshrun`` for SSH calls
3589 (instead of SSH directly).
3590 (`#5389 <https://github.com/datalad/datalad/issues/5389>`__)
3591
3592 - The ORA special remote has received several new features:
3593
3594 - It now support a ``push-url`` setting as an alternative to ``url``
3595 for write access. An analog parameter was also added to
3596 ``create-sibling-ria``.
3597 (`#5420 <https://github.com/datalad/datalad/issues/5420>`__,
3598 `#5428 <https://github.com/datalad/datalad/issues/5428>`__)
3599
3600 - Access of RIA stores now performs homogeneous availability checks,
3601 regardless of access protocol. Before, broken HTTP-based access
3602 due to misspecified URLs could have gone unnoticed.
3603 (`#5459 <https://github.com/datalad/datalad/issues/5459>`__,
3604 `#5672 <https://github.com/datalad/datalad/issues/5672>`__)
3605
3606 - Error reporting was introduce to inform about undesirable
3607 conditions in remote RIA stores.
3608 (`#5683 <https://github.com/datalad/datalad/issues/5683>`__)
3609
3610 - ``create-sibling-ria`` now supports ``--alias`` for the specification
3611 of a convenience dataset alias name in a RIA store.
3612 (`#5592 <https://github.com/datalad/datalad/issues/5592>`__)
3613
3614 - Analog to ``git commit``, ``save`` now features an ``--amend`` mode
3615 to support incremental updates of a dataset state.
3616 (`#5430 <https://github.com/datalad/datalad/issues/5430>`__)
3617
3618 - ``run`` now supports a dry-run mode that can be used to inspect the
3619 result of parameter expansion on the effective command to ease the
3620 composition of more complicated command lines.
3621 (`#5539 <https://github.com/datalad/datalad/issues/5539>`__)
3622
3623 - ``run`` now supports a ``--assume-ready`` switch to avoid the
3624 (possibly expensive) preparation of inputs and outputs with large
3625 datasets that have already been readied through other means.
3626 (`#5431 <https://github.com/datalad/datalad/issues/5431>`__)
3627
3628 - ``update`` now features ``--how`` and ``--how-subds`` parameters to
3629 configure how an update shall be performed. Supported modes are
3630 ``fetch`` (unchanged default), and ``merge`` (previously also
3631 possible via ``--merge``), but also new strategies like ``reset`` or
3632 ``checkout``.
3633 (`#5534 <https://github.com/datalad/datalad/issues/5534>`__)
3634
3635 - ``update`` has a new ``--follow=parentds-lazy`` mode that only
3636 performs a fetch operation in subdatasets when the desired commit is
3637 not yet present. During recursive updates involving many subdatasets
3638 this can substantially speed up performance.
3639 (`#5474 <https://github.com/datalad/datalad/issues/5474>`__)
3640
3641 - DataLad’s command line API can now report the version for individual
3642 commands via ``datalad <cmd> --version``. The output has been
3643 homogenized to ``<providing package> <version>``.
3644 (`#5543 <https://github.com/datalad/datalad/issues/5543>`__)
3645
3646 - ``create-sibling`` now logs information on an auto-generated sibling
3647 name, in the case that no ``--name/-s`` was provided.
3648 (`#5550 <https://github.com/datalad/datalad/issues/5550>`__)
3649
3650 - ``create-sibling-github`` has been updated to emit result records
3651 like any standard DataLad command. Previously it was implemented as a
3652 “plugin”, which did not support all standard API parameters.
3653 (`#5551 <https://github.com/datalad/datalad/issues/5551>`__)
3654
3655 - ``copy-file`` now also works with content-less files in datasets on
3656 crippled filesystems (adjusted mode), when a recent enough git-annex
3657 (8.20210428 or later) is available.
3658 (`#5630 <https://github.com/datalad/datalad/issues/5630>`__)
3659
3660 - ``addurls`` can now be instructed how to behave in the event of file
3661 name collision via a new parameter ``--on-collision``.
3662 (`#5675 <https://github.com/datalad/datalad/issues/5675>`__)
3663
3664 - ``addurls`` reporting now informs which particular subdatasets were
3665 created. (`#5689 <https://github.com/datalad/datalad/issues/5689>`__)
3666
3667 - Credentials can now be provided or overwritten via all means
3668 supported by ``ConfigManager``. Importantly,
3669 ``datalad.credential.<name>.<field>`` configuration settings and
3670 analog specification via environment variables are now supported
3671 (rather than custom environment variables only). Previous
3672 specification methods are still supported too.
3673 (`#5680 <https://github.com/datalad/datalad/issues/5680>`__)
3674
3675 - A new ``datalad.credentials.force-ask`` configuration flag can now be
3676 used to force re-entry of already known credentials. This simplifies
3677 credential updates without having to use an approach native to
3678 individual credential stores.
3679 (`#5777 <https://github.com/datalad/datalad/issues/5777>`__)
3680
3681 - Suppression of rendering repeated similar results is now configurable
3682 via the configuration switches
3683 ``datalad.ui.suppress-similar-results`` (bool), and
3684 ``datalad.ui.suppress-similar-results-threshold`` (int).
3685 (`#5681 <https://github.com/datalad/datalad/issues/5681>`__)
3686
3687 - The performance of ``status`` and similar functionality when
3688 determining local file availability has been improved.
3689 (`#5692 <https://github.com/datalad/datalad/issues/5692>`__)
3690
3691 - ``push`` now renders a result summary on completion.
3692 (`#5696 <https://github.com/datalad/datalad/issues/5696>`__)
3693
3694 - A dedicated info log message indicates when dataset repositories are
3695 subjected to an annex version upgrade.
3696 (`#5698 <https://github.com/datalad/datalad/issues/5698>`__)
3697
3698 - Error reporting improvements:
3699
3700 - The ``NoDatasetFound`` exception now provides information for
3701 which purpose a dataset is required.
3702 (`#5708 <https://github.com/datalad/datalad/issues/5708>`__)
3703
3704 - Wording of the ``MissingExternalDependeny`` error was rephrased to
3705 account for cases of non-functional installations.
3706 (`#5803 <https://github.com/datalad/datalad/issues/5803>`__)
3707
3708 - ``push`` reports when a ``--to`` parameter specification was
3709 (likely) forgotten.
3710 (`#5726 <https://github.com/datalad/datalad/issues/5726>`__)
3711
3712 - Detailed information is now given when DataLad fails to obtain a
3713 lock for credential entry in a timely fashion. Previously only a
3714 generic debug log message was emitted.
3715 (`#5884 <https://github.com/datalad/datalad/issues/5884>`__)
3716
3717 - Clarified error message when ``create-sibling-gitlab`` was called
3718 without ``--project``.
3719 (`#5907 <https://github.com/datalad/datalad/issues/5907>`__)
3720
3721 - ``add-readme`` now provides a README template with more information
3722 on the nature and use of DataLad datasets. A README file is no longer
3723 annex’ed by default, but can be using the new ``--annex`` switch.
3724 ([#5723][], [#5725][])
3725
3726 - ``clean`` now supports a ``--dry-run`` mode to inform about cleanable
3727 content. (`#5738 <https://github.com/datalad/datalad/issues/5738>`__)
3728
3729 - A new configuration setting ``datalad.locations.locks`` can be used
3730 to control the placement of lock files.
3731 (`#5740 <https://github.com/datalad/datalad/issues/5740>`__)
3732
3733 - ``wtf`` now also reports branch names and states.
3734 (`#5804 <https://github.com/datalad/datalad/issues/5804>`__)
3735
3736 - ``AnnexRepo.whereis()`` now supports batch mode.
3737 (`#5533 <https://github.com/datalad/datalad/issues/5533>`__)
3738
3739 .. _deprecations-and-removals-2:
3740
3741 Deprecations and removals
3742 ~~~~~~~~~~~~~~~~~~~~~~~~~
3743
3744 - The minimum supported git-annex version is now 8.20200309.
3745 (`#5512 <https://github.com/datalad/datalad/issues/5512>`__)
3746
3747 - ORA special remote configuration items ``ssh-host``, and
3748 ``base-path`` are deprecated. They are completely replaced by
3749 ``ria+<protocol>://`` URL specifications.
3750 (`#5425 <https://github.com/datalad/datalad/issues/5425>`__)
3751
3752 - The deprecated ``no_annex`` parameter of ``create()`` was removed
3753 from the Python API.
3754 (`#5441 <https://github.com/datalad/datalad/issues/5441>`__)
3755
3756 - The unused ``GitRepo.pull()`` method has been removed.
3757 (`#5558 <https://github.com/datalad/datalad/issues/5558>`__)
3758
3759 - Residual support for “plugins” (a mechanism used before DataLad
3760 supported extensions) was removed. This includes the configuration
3761 switches ``datalad.locations.{system,user}-plugins``.
3762 (`#5554 <https://github.com/datalad/datalad/issues/5554>`__,
3763 `#5564 <https://github.com/datalad/datalad/issues/5564>`__)
3764
3765 - Several features and comments have been moved to the
3766 ``datalad-deprecated`` package. This package must now be installed to
3767 be able to use keep using this functionality.
3768
3769 - The ``publish`` command. Use ``push`` instead.
3770 (`#5837 <https://github.com/datalad/datalad/issues/5837>`__)
3771
3772 - The ``ls`` command.
3773 (`#5569 <https://github.com/datalad/datalad/issues/5569>`__)
3774
3775 - The web UI that is deployable via ``datalad create-sibling --ui``.
3776 (`#5555 <https://github.com/datalad/datalad/issues/5555>`__)
3777
3778 - The “automagic IO” feature.
3779 (`#5577 <https://github.com/datalad/datalad/issues/5577>`__)
3780
3781 - ``AnnexRepo.copy_to()`` has been deprecated. The ``push`` command
3782 should be used instead.
3783 (`#5560 <https://github.com/datalad/datalad/issues/5560>`__)
3784
3785 - ``AnnexRepo.sync()`` has been deprecated.
3786 ``AnnexRepo.call_annex(['sync', ...])`` should be used instead.
3787 (`#5461 <https://github.com/datalad/datalad/issues/5461>`__)
3788
3789 - All ``GitRepo.*_submodule()`` methods have been deprecated and will
3790 be removed in a future release.
3791 (`#5559 <https://github.com/datalad/datalad/issues/5559>`__)
3792
3793 - ``create-sibling-github``\ ’s ``--dryrun`` switch was deprecated, use
3794 ``--dry-run`` instead.
3795 (`#5551 <https://github.com/datalad/datalad/issues/5551>`__)
3796
3797 - The ``datalad --pbs-runner`` option has been deprecated, use
3798 ``condor_run`` (or similar) instead.
3799 (`#5956 <https://github.com/datalad/datalad/issues/5956>`__)
3800
3801 Fixes
3802 -----
3803
3804 - Prevent invalid declaration of a publication dependencies for
3805 ‘origin’ on any auto-detected ORA special remotes, when cloing from a
3806 RIA store. An ORA remote is now checked whether it actually points to
3807 the RIA store the clone was made from.
3808 (`#5415 <https://github.com/datalad/datalad/issues/5415>`__)
3809
3810 - The ORA special remote implementation has received several fixes:
3811
3812 - It can now handle HTTP redirects.
3813 (`#5792 <https://github.com/datalad/datalad/issues/5792>`__)
3814
3815 - Prevents failure when URL-type annex keys contain the ‘/’
3816 character.
3817 (`#5823 <https://github.com/datalad/datalad/issues/5823>`__)
3818
3819 - Properly support the specification of usernames, passwords and
3820 ports in ``ria+<protocol>://`` URLs.
3821 (`#5902 <https://github.com/datalad/datalad/issues/5902>`__)
3822
3823 - It is now possible to specifically select the default (or generic)
3824 result renderer via ``datalad -f default`` and with that override a
3825 ``tailored`` result renderer that may be preconfigured for a
3826 particular command.
3827 (`#5476 <https://github.com/datalad/datalad/issues/5476>`__)
3828
3829 - Starting with 0.14.0, original URLs given to ``clone`` were recorded
3830 in a subdataset record. This was initially done in a second commit,
3831 leading to inflation of commits and slowdown in superdatasets with
3832 many subdatasets. Such subdataset record annotation is now collapsed
3833 into a single commits.
3834 (`#5480 <https://github.com/datalad/datalad/issues/5480>`__)
3835
3836 - ``run`` now longer removes leading empty directories as part of the
3837 output preparation. This was surprising behavior for commands that do
3838 not ensure on their own that output directories exist.
3839 (`#5492 <https://github.com/datalad/datalad/issues/5492>`__)
3840
3841 - A potentially existing ``message`` property is no longer removed when
3842 using the ``json`` or ``json_pp`` result renderer to avoid undesired
3843 withholding of relevant information.
3844 (`#5536 <https://github.com/datalad/datalad/issues/5536>`__)
3845
3846 - ``subdatasets`` now reports ``state=present``, rather than
3847 ``state=clean``, for installed subdatasets to complement
3848 ``state=absent`` reports for uninstalled dataset.
3849 (`#5655 <https://github.com/datalad/datalad/issues/5655>`__)
3850
3851 - ``create-sibling-ria`` now executes commands with a consistent
3852 environment setup that matches all other command execution in other
3853 DataLad commands.
3854 (`#5682 <https://github.com/datalad/datalad/issues/5682>`__)
3855
3856 - ``save`` no longer saves unspecified subdatasets when called with an
3857 explicit path (list). The fix required a behavior change of
3858 ``GitRepo.get_content_info()`` in its interpretation of ``None``
3859 vs. ``[]`` path argument values that now aligns the behavior of
3860 ``GitRepo.diff|status()`` with their respective documentation.
3861 (`#5693 <https://github.com/datalad/datalad/issues/5693>`__)
3862
3863 - ``get`` now prefers the location of a subdatasets that is recorded in
3864 a superdataset’s ``.gitmodules`` record. Previously, DataLad tried to
3865 obtain a subdataset from an assumed checkout of the superdataset’s
3866 origin. This new default order is (re-)configurable via the
3867 ``datalad.get.subdataset-source-candidate-<priority-label>``
3868 configuration mechanism.
3869 (`#5760 <https://github.com/datalad/datalad/issues/5760>`__)
3870
3871 - ``create-sibling-gitlab`` no longer skips the root dataset when ``.``
3872 is given as a path.
3873 (`#5789 <https://github.com/datalad/datalad/issues/5789>`__)
3874
3875 - ``siblings`` now rejects a value given to ``--as-common-datasrc``
3876 that clashes with the respective Git remote.
3877 (`#5805 <https://github.com/datalad/datalad/issues/5805>`__)
3878
3879 - The usage synopsis reported by ``siblings`` now lists all supported
3880 actions. (`#5913 <https://github.com/datalad/datalad/issues/5913>`__)
3881
3882 - ``siblings`` now renders non-ok results to avoid silent failure.
3883 (`#5915 <https://github.com/datalad/datalad/issues/5915>`__)
3884
3885 - ``.gitattribute`` file manipulations no longer leave the file without
3886 a trailing newline.
3887 (`#5847 <https://github.com/datalad/datalad/issues/5847>`__)
3888
3889 - Prevent crash when trying to delete a non-existing keyring credential
3890 field. (`#5892 <https://github.com/datalad/datalad/issues/5892>`__)
3891
3892 - git-annex is no longer called with an unconditional ``annex.retry=3``
3893 configuration. Instead, this parameterization is now limited to
3894 ``annex get`` and ``annex copy`` calls.
3895 (`#5904 <https://github.com/datalad/datalad/issues/5904>`__)
3896
3897 .. _tests-31:
3898
3899 Tests
3900 -----
3901
3902 - ``file://`` URLs are no longer the predominant test case for
3903 ``AnnexRepo`` functionality. A built-in HTTP server now used in most
3904 cases. (`#5332 <https://github.com/datalad/datalad/issues/5332>`__)
3905
3906 --------------
3907
3908 0.14.8 (Sun Sep 12 2021)
3909 ========================
3910
3911 .. _bug-fix-17:
3912
3913 Bug Fix
3914 -------
3915
3916 - BF: add-archive-content on .xz and other non-.gz stream compressed
3917 files `#5930 <https://github.com/datalad/datalad/pull/5930>`__
3918 (`@yarikoptic <https://github.com/yarikoptic>`__)
3919 - BF(UX): do not keep logging ERROR possibly present in progress
3920 records `#5936 <https://github.com/datalad/datalad/pull/5936>`__
3921 (`@yarikoptic <https://github.com/yarikoptic>`__)
3922 - Annotate datalad_core as not needing actual data – just uses annex
3923 whereis `#5971 <https://github.com/datalad/datalad/pull/5971>`__
3924 (`@yarikoptic <https://github.com/yarikoptic>`__)
3925 - BF: limit CMD_MAX_ARG if obnoxious value is encountered.
3926 `#5945 <https://github.com/datalad/datalad/pull/5945>`__
3927 (`@yarikoptic <https://github.com/yarikoptic>`__)
3928 - Download session/credentials locking – inform user if locking is
3929 “failing” to be obtained, fail upon ~5min timeout
3930 `#5884 <https://github.com/datalad/datalad/pull/5884>`__
3931 (`@yarikoptic <https://github.com/yarikoptic>`__)
3932 - Render siblings()’s non-ok results with the default renderer
3933 `#5915 <https://github.com/datalad/datalad/pull/5915>`__
3934 (`@mih <https://github.com/mih>`__)
3935 - BF: do not crash, just skip whenever trying to delete non existing
3936 field in the underlying keyring
3937 `#5892 <https://github.com/datalad/datalad/pull/5892>`__
3938 (`@yarikoptic <https://github.com/yarikoptic>`__)
3939 - Fix argument-spec for ``siblings`` and improve usage synopsis
3940 `#5913 <https://github.com/datalad/datalad/pull/5913>`__
3941 (`@mih <https://github.com/mih>`__)
3942 - Clarify error message re unspecified gitlab project
3943 `#5907 <https://github.com/datalad/datalad/pull/5907>`__
3944 (`@mih <https://github.com/mih>`__)
3945 - Support username, password and port specification in RIA URLs
3946 `#5902 <https://github.com/datalad/datalad/pull/5902>`__
3947 (`@mih <https://github.com/mih>`__)
3948 - BF: take path from SSHRI, test URLs not only on Windows
3949 `#5881 <https://github.com/datalad/datalad/pull/5881>`__
3950 (`@yarikoptic <https://github.com/yarikoptic>`__)
3951 - ENH(UX): warn user if keyring returned a “null” keyring
3952 `#5875 <https://github.com/datalad/datalad/pull/5875>`__
3953 (`@yarikoptic <https://github.com/yarikoptic>`__)
3954 - ENH(UX): state original purpose in NoDatasetFound exception + detail
3955 it for get `#5708 <https://github.com/datalad/datalad/pull/5708>`__
3956 (`@yarikoptic <https://github.com/yarikoptic>`__)
3957
3958 .. _pushed-to-maint-5:
3959
3960 Pushed to ``maint``
3961 -------------------
3962
3963 - Merge branch ‘bf-http-headers-agent’ into maint
3964 (`@yarikoptic <https://github.com/yarikoptic>`__)
3965 - RF(BF?)+DOC: provide User-Agent to entire session headers + use those
3966 if provided (`@yarikoptic <https://github.com/yarikoptic>`__)
3967
3968 .. _internal-23:
3969
3970 Internal
3971 --------
3972
3973 - Pass ``--no-changelog`` to ``auto shipit`` if changelog already has
3974 entry `#5952 <https://github.com/datalad/datalad/pull/5952>`__
3975 (`@jwodder <https://github.com/jwodder>`__)
3976 - Add isort config to match current convention + run isort via
3977 pre-commit (if configured)
3978 `#5923 <https://github.com/datalad/datalad/pull/5923>`__
3979 (`@jwodder <https://github.com/jwodder>`__)
3980 - .travis.yml: use python -m {nose,coverage} invocations, and always
3981 show combined report
3982 `#5888 <https://github.com/datalad/datalad/pull/5888>`__
3983 (`@yarikoptic <https://github.com/yarikoptic>`__)
3984 - Add project URLs into the package metadata for convenience links on
3985 Pypi `#5866 <https://github.com/datalad/datalad/pull/5866>`__
3986 (`@adswa <https://github.com/adswa>`__
3987 `@yarikoptic <https://github.com/yarikoptic>`__)
3988
3989 .. _tests-32:
3990
3991 Tests
3992 -----
3993
3994 - BF: do use OBSCURE_FILENAME instead of hardcoded unicode
3995 `#5944 <https://github.com/datalad/datalad/pull/5944>`__
3996 (`@yarikoptic <https://github.com/yarikoptic>`__)
3997 - BF(TST): Skip testing for having PID listed if no psutil
3998 `#5920 <https://github.com/datalad/datalad/pull/5920>`__
3999 (`@yarikoptic <https://github.com/yarikoptic>`__)
4000 - BF(TST): Boost version of git-annex to 8.20201129 to test an error
4001 message `#5894 <https://github.com/datalad/datalad/pull/5894>`__
4002 (`@yarikoptic <https://github.com/yarikoptic>`__)
4003
4004 .. _authors-4-1:
4005
4006 Authors: 4
4007 ----------
4008
4009 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
4010 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
4011 - Michael Hanke (`@mih <https://github.com/mih>`__)
4012 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
4013
4014 --------------
4015
4016 0.14.7 (Tue Aug 03 2021)
4017 ========================
4018
4019 .. _bug-fix-18:
4020
4021 Bug Fix
4022 -------
4023
4024 - UX: When two or more clone URL templates are found, error out more
4025 gracefully `#5839 <https://github.com/datalad/datalad/pull/5839>`__
4026 (`@adswa <https://github.com/adswa>`__)
4027 - BF: http_auth - follow redirect (just 1) to re-authenticate after
4028 initial attempt
4029 `#5852 <https://github.com/datalad/datalad/pull/5852>`__
4030 (`@yarikoptic <https://github.com/yarikoptic>`__)
4031 - addurls Formatter - provide value repr in exception
4032 `#5850 <https://github.com/datalad/datalad/pull/5850>`__
4033 (`@yarikoptic <https://github.com/yarikoptic>`__)
4034 - ENH: allow for “patch” level semver for “master” branch
4035 `#5839 <https://github.com/datalad/datalad/pull/5839>`__
4036 (`@yarikoptic <https://github.com/yarikoptic>`__)
4037 - BF: Report info from annex JSON error message in CommandError
4038 `#5809 <https://github.com/datalad/datalad/pull/5809>`__
4039 (`@mih <https://github.com/mih>`__)
4040 - RF(TST): do not test for no EASY and pkg_resources in shims
4041 `#5817 <https://github.com/datalad/datalad/pull/5817>`__
4042 (`@yarikoptic <https://github.com/yarikoptic>`__)
4043 - http downloaders: Provide custom informative User-Agent, do not claim
4044 to be “Authenticated access”
4045 `#5802 <https://github.com/datalad/datalad/pull/5802>`__
4046 (`@yarikoptic <https://github.com/yarikoptic>`__)
4047 - ENH(UX,DX): inform user with a warning if version is 0+unknown
4048 `#5787 <https://github.com/datalad/datalad/pull/5787>`__
4049 (`@yarikoptic <https://github.com/yarikoptic>`__)
4050 - shell-completion: add argcomplete to ‘misc’ extra_depends, log an
4051 ERROR if argcomplete fails to import
4052 `#5781 <https://github.com/datalad/datalad/pull/5781>`__
4053 (`@yarikoptic <https://github.com/yarikoptic>`__)
4054 - ENH (UX): add python-gitlab dependency
4055 `#5776 <https://github.com/datalad/datalad/pull/5776>`__
4056 ([email protected])
4057
4058 .. _internal-24:
4059
4060 Internal
4061 --------
4062
4063 - BF: Fix reported paths in ORA remote
4064 `#5821 <https://github.com/datalad/datalad/pull/5821>`__
4065 (`@adswa <https://github.com/adswa>`__)
4066 - BF: import importlib.metadata not importlib_metadata whenever
4067 available `#5818 <https://github.com/datalad/datalad/pull/5818>`__
4068 (`@yarikoptic <https://github.com/yarikoptic>`__)
4069
4070 .. _tests-33:
4071
4072 Tests
4073 -----
4074
4075 - TST: set –allow-unrelated-histories in the mk_push_target setup for
4076 Windows `#5855 <https://github.com/datalad/datalad/pull/5855>`__
4077 (`@adswa <https://github.com/adswa>`__)
4078 - Tests: Allow for version to contain + as a separator and provide more
4079 information for version related comparisons
4080 `#5786 <https://github.com/datalad/datalad/pull/5786>`__
4081 (`@yarikoptic <https://github.com/yarikoptic>`__)
4082
4083 .. _authors-4-2:
4084
4085 Authors: 4
4086 ----------
4087
4088 - Adina Wagner (`@adswa <https://github.com/adswa>`__)
4089 - Michael Hanke (`@mih <https://github.com/mih>`__)
4090 - Stephan Heunis (`@jsheunis <https://github.com/jsheunis>`__)
4091 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
4092
4093 --------------
4094
4095 0.14.6 (Sun Jun 27 2021)
4096 ========================
4097
4098 .. _internal-25:
4099
4100 Internal
4101 --------
4102
4103 - BF: update changelog conversion from .md to .rst (for sphinx)
4104 `#5757 <https://github.com/datalad/datalad/pull/5757>`__
4105 (`@yarikoptic <https://github.com/yarikoptic>`__
4106 `@jwodder <https://github.com/jwodder>`__)
4107
4108 .. _authors-2-1:
4109
4110 Authors: 2
4111 ----------
4112
4113 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
4114 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
4115
4116 --------------
4117
4118 0.14.5 (Mon Jun 21 2021)
4119 ========================
4120
4121 .. _bug-fix-19:
4122
4123 Bug Fix
4124 -------
4125
4126 - BF(TST): parallel - take longer for producer to produce
4127 `#5747 <https://github.com/datalad/datalad/pull/5747>`__
4128 (`@yarikoptic <https://github.com/yarikoptic>`__)
4129 - add –on-failure default value and document it
4130 `#5690 <https://github.com/datalad/datalad/pull/5690>`__
4131 (`@christian-monch <https://github.com/christian-monch>`__
4132 `@yarikoptic <https://github.com/yarikoptic>`__)
4133 - ENH: harmonize “purpose” statements to imperative form
4134 `#5733 <https://github.com/datalad/datalad/pull/5733>`__
4135 (`@yarikoptic <https://github.com/yarikoptic>`__)
4136 - ENH(TST): populate heavy tree with 100 unique keys (not just 1) among
4137 10,000 `#5734 <https://github.com/datalad/datalad/pull/5734>`__
4138 (`@yarikoptic <https://github.com/yarikoptic>`__)
4139 - BF: do not use .acquired - just get state from acquire()
4140 `#5718 <https://github.com/datalad/datalad/pull/5718>`__
4141 (`@yarikoptic <https://github.com/yarikoptic>`__)
4142 - BF: account for annex now “scanning for annexed” instead of
4143 “unlocked” files
4144 `#5705 <https://github.com/datalad/datalad/pull/5705>`__
4145 (`@yarikoptic <https://github.com/yarikoptic>`__)
4146 - interface: Don’t repeat custom summary for non-generator results
4147 `#5688 <https://github.com/datalad/datalad/pull/5688>`__
4148 (`@kyleam <https://github.com/kyleam>`__)
4149 - RF: just pip install datalad-installer
4150 `#5676 <https://github.com/datalad/datalad/pull/5676>`__
4151 (`@yarikoptic <https://github.com/yarikoptic>`__)
4152 - DOC: addurls.extract: Drop mention of removed ‘stream’ parameter
4153 `#5690 <https://github.com/datalad/datalad/pull/5690>`__
4154 (`@kyleam <https://github.com/kyleam>`__)
4155 - Merge pull request #5674 from kyleam/test-addurls-copy-fix
4156 `#5674 <https://github.com/datalad/datalad/pull/5674>`__
4157 (`@kyleam <https://github.com/kyleam>`__)
4158 - Merge pull request #5663 from kyleam/status-ds-equal-path
4159 `#5663 <https://github.com/datalad/datalad/pull/5663>`__
4160 (`@kyleam <https://github.com/kyleam>`__)
4161 - Merge pull request #5671 from kyleam/update-fetch-fail
4162 `#5671 <https://github.com/datalad/datalad/pull/5671>`__
4163 (`@kyleam <https://github.com/kyleam>`__)
4164 - BF: update: Honor –on-failure if fetch fails
4165 `#5671 <https://github.com/datalad/datalad/pull/5671>`__
4166 (`@kyleam <https://github.com/kyleam>`__)
4167 - RF: update: Avoid fetch’s deprecated kwargs
4168 `#5671 <https://github.com/datalad/datalad/pull/5671>`__
4169 (`@kyleam <https://github.com/kyleam>`__)
4170 - CLN: update: Drop an unused import
4171 `#5671 <https://github.com/datalad/datalad/pull/5671>`__
4172 (`@kyleam <https://github.com/kyleam>`__)
4173 - Merge pull request #5664 from kyleam/addurls-better-url-parts-error
4174 `#5664 <https://github.com/datalad/datalad/pull/5664>`__
4175 (`@kyleam <https://github.com/kyleam>`__)
4176 - Merge pull request #5661 from kyleam/sphinx-fix-plugin-refs
4177 `#5661 <https://github.com/datalad/datalad/pull/5661>`__
4178 (`@kyleam <https://github.com/kyleam>`__)
4179 - BF: status: Provide special treatment of “this dataset” path
4180 `#5663 <https://github.com/datalad/datalad/pull/5663>`__
4181 (`@kyleam <https://github.com/kyleam>`__)
4182 - BF: addurls: Provide better placeholder error for special keys
4183 `#5664 <https://github.com/datalad/datalad/pull/5664>`__
4184 (`@kyleam <https://github.com/kyleam>`__)
4185 - RF: addurls: Simply construction of placeholder exception message
4186 `#5664 <https://github.com/datalad/datalad/pull/5664>`__
4187 (`@kyleam <https://github.com/kyleam>`__)
4188 - RF: addurls._get_placeholder_exception: Rename a parameter
4189 `#5664 <https://github.com/datalad/datalad/pull/5664>`__
4190 (`@kyleam <https://github.com/kyleam>`__)
4191 - RF: status: Avoid repeated Dataset.path access
4192 `#5663 <https://github.com/datalad/datalad/pull/5663>`__
4193 (`@kyleam <https://github.com/kyleam>`__)
4194 - DOC: Reference plugins via datalad.api
4195 `#5661 <https://github.com/datalad/datalad/pull/5661>`__
4196 (`@kyleam <https://github.com/kyleam>`__)
4197 - download-url: Set up datalad special remote if needed
4198 `#5648 <https://github.com/datalad/datalad/pull/5648>`__
4199 (`@kyleam <https://github.com/kyleam>`__
4200 `@yarikoptic <https://github.com/yarikoptic>`__)
4201
4202 .. _pushed-to-maint-6:
4203
4204 Pushed to ``maint``
4205 -------------------
4206
4207 - MNT: Post-release dance (`@kyleam <https://github.com/kyleam>`__)
4208
4209 .. _internal-26:
4210
4211 Internal
4212 --------
4213
4214 - Switch to versioneer and auto
4215 `#5669 <https://github.com/datalad/datalad/pull/5669>`__
4216 (`@jwodder <https://github.com/jwodder>`__
4217 `@yarikoptic <https://github.com/yarikoptic>`__)
4218 - MNT: setup.py: Temporarily avoid Sphinx 4
4219 `#5649 <https://github.com/datalad/datalad/pull/5649>`__
4220 (`@kyleam <https://github.com/kyleam>`__)
4221
4222 .. _tests-34:
4223
4224 Tests
4225 -----
4226
4227 - BF(TST): skip testing for showing “Scanning for …” since not shown if
4228 too quick `#5727 <https://github.com/datalad/datalad/pull/5727>`__
4229 (`@yarikoptic <https://github.com/yarikoptic>`__)
4230 - Revert “TST: test_partial_unlocked: Document and avoid recent
4231 git-annex failure”
4232 `#5651 <https://github.com/datalad/datalad/pull/5651>`__
4233 (`@kyleam <https://github.com/kyleam>`__)
4234
4235 .. _authors-4-3:
4236
4237 Authors: 4
4238 ----------
4239
4240 - Christian Mnch
4241 (`@christian-monch <https://github.com/christian-monch>`__)
4242 - John T. Wodder II (`@jwodder <https://github.com/jwodder>`__)
4243 - Kyle Meyer (`@kyleam <https://github.com/kyleam>`__)
4244 - Yaroslav Halchenko (`@yarikoptic <https://github.com/yarikoptic>`__)
4245
4246 --------------
4247
4248 0.14.4 (May 10, 2021) – .
4249 =========================
4250
4251 .. _fixes-1:
4252
4253 Fixes
4254 -----
4255
4256 - Following an internal call to ``git-clone``,
4257 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
4258 assumed that the remote name was “origin”, but this may not be the
4259 case if ``clone.defaultRemoteName`` is configured (available as of
4260 Git 2.30).
4261 (`#5572 <https://github.com/datalad/datalad/issues/5572>`__)
4262
4263 - Several test fixes, including updates for changes in git-annex.
4264 (`#5612 <https://github.com/datalad/datalad/issues/5612>`__)
4265 (`#5632 <https://github.com/datalad/datalad/issues/5632>`__)
4266 (`#5639 <https://github.com/datalad/datalad/issues/5639>`__)
4267
4268 0.14.3 (April 28, 2021) – .
4269 ===========================
4270
4271 .. _fixes-2:
4272
4273 Fixes
4274 -----
4275
4276 - For outputs that include a glob,
4277 `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
4278 didn’t re-glob after executing the command, which is necessary to
4279 catch changes if ``--explicit`` or ``--expand={outputs,both}`` is
4280 specified.
4281 (`#5594 <https://github.com/datalad/datalad/issues/5594>`__)
4282
4283 - `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
4284 now gives an error result rather than a warning when an input glob
4285 doesn’t match.
4286 (`#5594 <https://github.com/datalad/datalad/issues/5594>`__)
4287
4288 - The procedure for creating a RIA store checks for an existing
4289 ria-layout-version file and makes sure its version matches the
4290 desired version. This check wasn’t done correctly for SSH hosts.
4291 (`#5607 <https://github.com/datalad/datalad/issues/5607>`__)
4292
4293 - A helper for transforming git-annex JSON records into DataLad results
4294 didn’t account for the unusual case where the git-annex record
4295 doesn’t have a “file” key.
4296 (`#5580 <https://github.com/datalad/datalad/issues/5580>`__)
4297
4298 - The test suite required updates for recent changes in PyGithub and
4299 git-annex.
4300 (`#5603 <https://github.com/datalad/datalad/issues/5603>`__)
4301 (`#5609 <https://github.com/datalad/datalad/issues/5609>`__)
4302
4303 .. _enhancements-and-new-features-10:
4304
4305 Enhancements and new features
4306 -----------------------------
4307
4308 - The DataLad source repository has long had a tools/cmdline-completion
4309 helper. This functionality is now exposed as a command,
4310 ``datalad shell-completion``.
4311 (`#5544 <https://github.com/datalad/datalad/issues/5544>`__)
4312
4313 0.14.2 (April 14, 2021) – .
4314 ===========================
4315
4316 .. _fixes-3:
4317
4318 Fixes
4319 -----
4320
4321 - `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__
4322 now works bottom-up, pushing submodules first so that hooks on the
4323 remote can aggregate updated subdataset information.
4324 (`#5416 <https://github.com/datalad/datalad/issues/5416>`__)
4325
4326 - `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
4327 didn’t ensure that the configuration of subdatasets was reloaded.
4328 (`#5552 <https://github.com/datalad/datalad/issues/5552>`__)
4329
4330 0.14.1 (April 01, 2021) – .
4331 ===========================
4332
4333 .. _fixes-4:
4334
4335 Fixes
4336 -----
4337
4338 - The recent default branch changes on GitHub’s side can lead to
4339 “git-annex” being selected over “master” as the default branch on
4340 GitHub when setting up a sibling with
4341 `create-sibling-github <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-github.html>`__.
4342 To work around this, the current branch is now pushed first.
4343 (`#5010 <https://github.com/datalad/datalad/issues/5010>`__)
4344
4345 - The logic for reading in a JSON line from git-annex failed if the
4346 response exceeded the buffer size (256 KB on \*nix systems).
4347
4348 - Calling
4349 `unlock <http://datalad.readthedocs.io/en/latest/generated/man/datalad-unlock.html>`__
4350 with a path of “.” from within an untracked subdataset incorrectly
4351 aborted, complaining that the “dataset containing given paths is not
4352 underneath the reference dataset”.
4353 (`#5458 <https://github.com/datalad/datalad/issues/5458>`__)
4354
4355 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
4356 didn’t account for the possibility of multiple accessible ORA remotes
4357 or the fact that none of them may be associated with the RIA store
4358 being cloned.
4359 (`#5488 <https://github.com/datalad/datalad/issues/5488>`__)
4360
4361 - `create-sibling-ria <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-ria.html>`__
4362 didn’t call ``git update-server-info`` after setting up the remote
4363 repository and, as a result, the repository couldn’t be fetched until
4364 something else (e.g., a push) triggered a call to
4365 ``git update-server-info``.
4366 (`#5531 <https://github.com/datalad/datalad/issues/5531>`__)
4367
4368 - The parser for git-config output didn’t properly handle multi-line
4369 values and got thrown off by unexpected and unrelated lines.
4370 (`#5509 <https://github.com/datalad/datalad/issues/5509>`__)
4371
4372 - The 0.14 release introduced regressions in the handling of progress
4373 bars for git-annex actions, including collapsing progress bars for
4374 concurrent operations.
4375 (`#5421 <https://github.com/datalad/datalad/issues/5421>`__)
4376 (`#5438 <https://github.com/datalad/datalad/issues/5438>`__)
4377
4378 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
4379 failed if the user configured Git’s ``diff.ignoreSubmodules`` to a
4380 non-default value.
4381 (`#5453 <https://github.com/datalad/datalad/issues/5453>`__)
4382
4383 - A interprocess lock is now used to prevent a race between checking
4384 for an SSH socket’s existence and creating it.
4385 (`#5466 <https://github.com/datalad/datalad/issues/5466>`__)
4386
4387 - If a Python procedure script is executable,
4388 `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
4389 invokes it directly rather than passing it to ``sys.executable``. The
4390 non-executable Python procedures that ship with DataLad now include
4391 shebangs so that invoking them has a chance of working on file
4392 systems that present all files as executable.
4393 (`#5436 <https://github.com/datalad/datalad/issues/5436>`__)
4394
4395 - DataLad’s wrapper around ``argparse`` failed if an underscore was
4396 used in a positional argument.
4397 (`#5525 <https://github.com/datalad/datalad/issues/5525>`__)
4398
4399 .. _enhancements-and-new-features-11:
4400
4401 Enhancements and new features
4402 -----------------------------
4403
4404 - DataLad’s method for mapping environment variables to configuration
4405 options (e.g., ``DATALAD_FOO_X__Y`` to ``datalad.foo.x-y``) doesn’t
4406 work if the subsection name (“FOO”) has an underscore. This
4407 limitation can be sidestepped with the new
4408 ``DATALAD_CONFIG_OVERRIDES_JSON`` environment variable, which can be
4409 set to a JSON record of configuration values.
4410 (`#5505 <https://github.com/datalad/datalad/issues/5505>`__)
4411
4412 0.14.0 (February 02, 2021) – .
4413 ==============================
4414
4415 Major refactoring and deprecations
4416 ----------------------------------
4417
4418 - Git versions below v2.19.1 are no longer supported.
4419 (`#4650 <https://github.com/datalad/datalad/issues/4650>`__)
4420
4421 - The minimum git-annex version is still 7.20190503, but, if you’re on
4422 Windows (or use adjusted branches in general), please upgrade to at
4423 least 8.20200330 but ideally 8.20210127 to get subdataset-related
4424 fixes. (`#4292 <https://github.com/datalad/datalad/issues/4292>`__)
4425 (`#5290 <https://github.com/datalad/datalad/issues/5290>`__)
4426
4427 - The minimum supported version of Python is now 3.6.
4428 (`#4879 <https://github.com/datalad/datalad/issues/4879>`__)
4429
4430 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
4431 is now deprecated in favor of
4432 `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__.
4433 It will be removed in the 0.15.0 release at the earliest.
4434
4435 - A new command runner was added in v0.13. Functionality related to the
4436 old runner has now been removed: ``Runner``, ``GitRunner``, and
4437 ``run_gitcommand_on_file_list_chunks`` from the ``datalad.cmd``
4438 module along with the ``datalad.tests.protocolremote``,
4439 ``datalad.cmd.protocol``, and ``datalad.cmd.protocol.prefix``
4440 configuration options.
4441 (`#5229 <https://github.com/datalad/datalad/issues/5229>`__)
4442
4443 - The ``--no-storage-sibling`` switch of ``create-sibling-ria`` is
4444 deprecated in favor of ``--storage-sibling=off`` and will be removed
4445 in a later release.
4446 (`#5090 <https://github.com/datalad/datalad/issues/5090>`__)
4447
4448 - The ``get_git_dir`` static method of ``GitRepo`` is deprecated and
4449 will be removed in a later release. Use the ``dot_git`` attribute of
4450 an instance instead.
4451 (`#4597 <https://github.com/datalad/datalad/issues/4597>`__)
4452
4453 - The ``ProcessAnnexProgressIndicators`` helper from
4454 ``datalad.support.annexrepo`` has been removed.
4455 (`#5259 <https://github.com/datalad/datalad/issues/5259>`__)
4456
4457 - The ``save`` argument of
4458 `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__,
4459 a noop since v0.6.0, has been dropped.
4460 (`#5278 <https://github.com/datalad/datalad/issues/5278>`__)
4461
4462 - The ``get_URLS`` method of ``AnnexCustomRemote`` is deprecated and
4463 will be removed in a later release.
4464 (`#4955 <https://github.com/datalad/datalad/issues/4955>`__)
4465
4466 - ``ConfigManager.get`` now returns a single value rather than a tuple
4467 when there are multiple values for the same key, as very few callers
4468 correctly accounted for the possibility of a tuple return value.
4469 Callers can restore the old behavior by passing ``get_all=True``.
4470 (`#4924 <https://github.com/datalad/datalad/issues/4924>`__)
4471
4472 - In 0.12.0, all of the ``assure_*`` functions in ``datalad.utils``
4473 were renamed as ``ensure_*``, keeping the old names around as
4474 compatibility aliases. The ``assure_*`` variants are now marked as
4475 deprecated and will be removed in a later release.
4476 (`#4908 <https://github.com/datalad/datalad/issues/4908>`__)
4477
4478 - The ``datalad.interface.run`` module, which was deprecated in 0.12.0
4479 and kept as a compatibility shim for ``datalad.core.local.run``, has
4480 been removed.
4481 (`#4583 <https://github.com/datalad/datalad/issues/4583>`__)
4482
4483 - The ``saver`` argument of ``datalad.core.local.run.run_command``,
4484 marked as obsolete in 0.12.0, has been removed.
4485 (`#4583 <https://github.com/datalad/datalad/issues/4583>`__)
4486
4487 - The ``dataset_only`` argument of the ``ConfigManager`` class was
4488 deprecated in 0.12 and has now been removed.
4489 (`#4828 <https://github.com/datalad/datalad/issues/4828>`__)
4490
4491 - The ``linux_distribution_name``, ``linux_distribution_release``, and
4492 ``on_debian_wheezy`` attributes in ``datalad.utils`` are no longer
4493 set at import time and will be removed in a later release. Use
4494 ``datalad.utils.get_linux_distribution`` instead.
4495 (`#4696 <https://github.com/datalad/datalad/issues/4696>`__)
4496
4497 - ``datalad.distribution.clone``, which was marked as obsolete in v0.12
4498 in favor of ``datalad.core.distributed.clone``, has been removed.
4499 (`#4904 <https://github.com/datalad/datalad/issues/4904>`__)
4500
4501 - ``datalad.support.annexrepo.N_AUTO_JOBS``, announced as deprecated in
4502 v0.12.6, has been removed.
4503 (`#4904 <https://github.com/datalad/datalad/issues/4904>`__)
4504
4505 - The ``compat`` parameter of ``GitRepo.get_submodules``, added in
4506 v0.12 as a temporary compatibility layer, has been removed.
4507 (`#4904 <https://github.com/datalad/datalad/issues/4904>`__)
4508
4509 - The long-deprecated (and non-functional) ``url`` parameter of
4510 ``GitRepo.__init__`` has been removed.
4511 (`#5342 <https://github.com/datalad/datalad/issues/5342>`__)
4512
4513 .. _fixes-5:
4514
4515 Fixes
4516 -----
4517
4518 - Cloning onto a system that enters adjusted branches by default (as
4519 Windows does) did not properly record the clone URL.
4520 (`#5128 <https://github.com/datalad/datalad/issues/5128>`__)
4521
4522 - The RIA-specific handling after calling
4523 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
4524 was correctly triggered by ``ria+http`` URLs but not ``ria+https``
4525 URLs. (`#4977 <https://github.com/datalad/datalad/issues/4977>`__)
4526
4527 - If the registered commit wasn’t found when cloning a subdataset, the
4528 failed attempt was left around.
4529 (`#5391 <https://github.com/datalad/datalad/issues/5391>`__)
4530
4531 - The remote calls to ``cp`` and ``chmod`` in
4532 `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
4533 were not portable and failed on macOS.
4534 (`#5108 <https://github.com/datalad/datalad/issues/5108>`__)
4535
4536 - A more reliable check is now done to decide if configuration files
4537 need to be reloaded.
4538 (`#5276 <https://github.com/datalad/datalad/issues/5276>`__)
4539
4540 - The internal command runner’s handling of the event loop has been
4541 improved to play nicer with outside applications and scripts that use
4542 asyncio. (`#5350 <https://github.com/datalad/datalad/issues/5350>`__)
4543 (`#5367 <https://github.com/datalad/datalad/issues/5367>`__)
4544
4545 .. _enhancements-and-new-features-12:
4546
4547 Enhancements and new features
4548 -----------------------------
4549
4550 - The subdataset handling for adjusted branches, which is particularly
4551 important on Windows where git-annex enters an adjusted branch by
4552 default, has been improved. A core piece of the new approach is
4553 registering the commit of the primary branch, not its checked out
4554 adjusted branch, in the superdataset. Note: This means that
4555 ``git status`` will always consider a subdataset on an adjusted
4556 branch as dirty while ``datalad status`` will look more closely and
4557 see if the tip of the primary branch matches the registered commit.
4558 (`#5241 <https://github.com/datalad/datalad/issues/5241>`__)
4559
4560 - The performance of the
4561 `subdatasets <http://datalad.readthedocs.io/en/latest/generated/man/datalad-subdatasets.html>`__
4562 command has been improved, with substantial speedups for recursive
4563 processing of many subdatasets.
4564 (`#4868 <https://github.com/datalad/datalad/issues/4868>`__)
4565 (`#5076 <https://github.com/datalad/datalad/issues/5076>`__)
4566
4567 - Adding new subdatasets via
4568 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
4569 has been sped up.
4570 (`#4793 <https://github.com/datalad/datalad/issues/4793>`__)
4571
4572 - `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__,
4573 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__,
4574 and
4575 `addurls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-addurls.html>`__
4576 gained support for parallel operations that can be enabled via the
4577 ``--jobs`` command-line option or the new
4578 ``datalad.runtime.max-jobs`` configuration option.
4579 (`#5022 <https://github.com/datalad/datalad/issues/5022>`__)
4580
4581 - `addurls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-addurls.html>`__
4582
4583 - learned how to read data from standard input.
4584 (`#4669 <https://github.com/datalad/datalad/issues/4669>`__)
4585 - now supports tab-separated input.
4586 (`#4845 <https://github.com/datalad/datalad/issues/4845>`__)
4587 - now lets Python callers pass in a list of records rather than a
4588 file name.
4589 (`#5285 <https://github.com/datalad/datalad/issues/5285>`__)
4590 - gained a ``--drop-after`` switch that signals to drop a file’s
4591 content after downloading and adding it to the annex.
4592 (`#5081 <https://github.com/datalad/datalad/issues/5081>`__)
4593 - is now able to construct a tree of files from known checksums
4594 without downloading content via its new ``--key`` option.
4595 (`#5184 <https://github.com/datalad/datalad/issues/5184>`__)
4596 - records the URL file in the commit message as provided by the
4597 caller rather than using the resolved absolute path.
4598 (`#5091 <https://github.com/datalad/datalad/issues/5091>`__)
4599 - is now speedier.
4600 (`#4867 <https://github.com/datalad/datalad/issues/4867>`__)
4601 (`#5022 <https://github.com/datalad/datalad/issues/5022>`__)
4602
4603 - `create-sibling-github <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-github.html>`__
4604 learned how to create private repositories (thanks to Nolan Nichols).
4605 (`#4769 <https://github.com/datalad/datalad/issues/4769>`__)
4606
4607 - `create-sibling-ria <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-ria.html>`__
4608 gained a ``--storage-sibling`` option. When
4609 ``--storage-sibling=only`` is specified, the storage sibling is
4610 created without an accompanying Git sibling. This enables using hosts
4611 without Git installed for storage.
4612 (`#5090 <https://github.com/datalad/datalad/issues/5090>`__)
4613
4614 - The download machinery (and thus the ``datalad`` special remote)
4615 gained support for a new scheme, ``shub://``, which follows the same
4616 format used by ``singularity run`` and friends. In contrast to the
4617 short-lived URLs obtained by querying Singularity Hub directly,
4618 ``shub://`` URLs are suitable for registering with git-annex.
4619 (`#4816 <https://github.com/datalad/datalad/issues/4816>`__)
4620
4621 - A provider is now included for https://registry-1.docker.io URLs.
4622 This is useful for storing an image’s blobs in a dataset and
4623 registering the URLs with git-annex.
4624 (`#5129 <https://github.com/datalad/datalad/issues/5129>`__)
4625
4626 - The ``add-readme`` command now links to the `DataLad
4627 handbook <http://handbook.datalad.org>`__ rather than
4628 http://docs.datalad.org.
4629 (`#4991 <https://github.com/datalad/datalad/issues/4991>`__)
4630
4631 - New option ``datalad.locations.extra-procedures`` specifies an
4632 additional location that should be searched for procedures.
4633 (`#5156 <https://github.com/datalad/datalad/issues/5156>`__)
4634
4635 - The class for handling configuration values, ``ConfigManager``, now
4636 takes a lock before writes to allow for multiple processes to modify
4637 the configuration of a dataset.
4638 (`#4829 <https://github.com/datalad/datalad/issues/4829>`__)
4639
4640 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
4641 now records the original, unresolved URL for a subdataset under
4642 ``submodule.<name>.datalad-url`` in the parent’s .gitmodules,
4643 enabling later
4644 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
4645 calls to use the original URL. This is particularly useful for
4646 ``ria+`` URLs.
4647 (`#5346 <https://github.com/datalad/datalad/issues/5346>`__)
4648
4649 - Installing a subdataset now uses custom handling rather than calling
4650 ``git submodule update --init``. This avoids some locking issues when
4651 running
4652 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
4653 in parallel and enables more accurate source URLs to be recorded.
4654 (`#4853 <https://github.com/datalad/datalad/issues/4853>`__)
4655
4656 - ``GitRepo.get_content_info``, a helper that gets triggered by many
4657 commands, got faster by tweaking its ``git ls-files`` call.
4658 (`#5067 <https://github.com/datalad/datalad/issues/5067>`__)
4659
4660 - `wtf <http://datalad.readthedocs.io/en/latest/generated/man/datalad-wtf.html>`__
4661 now includes credentials-related information (e.g. active backends)
4662 in the its output.
4663 (`#4982 <https://github.com/datalad/datalad/issues/4982>`__)
4664
4665 - The ``call_git*`` methods of ``GitRepo`` now have a ``read_only``
4666 parameter. Callers can set this to ``True`` to promise that the
4667 provided command does not write to the repository, bypassing the cost
4668 of some checks and locking.
4669 (`#5070 <https://github.com/datalad/datalad/issues/5070>`__)
4670
4671 - New ``call_annex*`` methods in the ``AnnexRepo`` class provide an
4672 interface for running git-annex commands similar to that of the
4673 ``GitRepo.call_git*`` methods.
4674 (`#5163 <https://github.com/datalad/datalad/issues/5163>`__)
4675
4676 - It’s now possible to register a custom metadata indexer that is
4677 discovered by
4678 `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
4679 and used to generate an index.
4680 (`#4963 <https://github.com/datalad/datalad/issues/4963>`__)
4681
4682 - The ``ConfigManager`` methods ``get``, ``getbool``, ``getfloat``, and
4683 ``getint`` now return a single value (with same precedence as
4684 ``git config --get``) when there are multiple values for the same
4685 key (in the non-committed git configuration, if the key is present
4686 there, or in the dataset configuration). For ``get``, the old
4687 behavior can be restored by specifying ``get_all=True``.
4688 (`#4924 <https://github.com/datalad/datalad/issues/4924>`__)
4689
4690 - Command-line scripts are now defined via the ``entry_points``
4691 argument of ``setuptools.setup`` instead of the ``scripts`` argument.
4692 (`#4695 <https://github.com/datalad/datalad/issues/4695>`__)
4693
4694 - Interactive use of ``--help`` on the command-line now invokes a pager
4695 on more systems and installation setups.
4696 (`#5344 <https://github.com/datalad/datalad/issues/5344>`__)
4697
4698 - The ``datalad`` special remote now tries to eliminate some
4699 unnecessary interactions with git-annex by being smarter about how it
4700 queries for URLs associated with a key.
4701 (`#4955 <https://github.com/datalad/datalad/issues/4955>`__)
4702
4703 - The ``GitRepo`` class now does a better job of handling bare
4704 repositories, a step towards bare repositories support in DataLad.
4705 (`#4911 <https://github.com/datalad/datalad/issues/4911>`__)
4706
4707 - More internal work to move the code base over to the new command
4708 runner. (`#4699 <https://github.com/datalad/datalad/issues/4699>`__)
4709 (`#4855 <https://github.com/datalad/datalad/issues/4855>`__)
4710 (`#4900 <https://github.com/datalad/datalad/issues/4900>`__)
4711 (`#4996 <https://github.com/datalad/datalad/issues/4996>`__)
4712 (`#5002 <https://github.com/datalad/datalad/issues/5002>`__)
4713 (`#5141 <https://github.com/datalad/datalad/issues/5141>`__)
4714 (`#5142 <https://github.com/datalad/datalad/issues/5142>`__)
4715 (`#5229 <https://github.com/datalad/datalad/issues/5229>`__)
4716
4717 0.13.7 (January 04, 2021) – .
4718 =============================
4719
4720 .. _fixes-6:
4721
4722 Fixes
4723 -----
4724
4725 - Cloning from a RIA store on the local file system initialized annex
4726 in the Git sibling of the RIA source, which is problematic because
4727 all annex-related functionality should go through the storage
4728 sibling.
4729 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
4730 now sets ``remote.origin.annex-ignore`` to ``true`` after cloning
4731 from RIA stores to prevent this.
4732 (`#5255 <https://github.com/datalad/datalad/issues/5255>`__)
4733
4734 - `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
4735 invoked ``cp`` in a way that was not compatible with macOS.
4736 (`#5269 <https://github.com/datalad/datalad/issues/5269>`__)
4737
4738 - Due to a bug in older Git versions (before 2.25), calling
4739 `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__
4740 with a file under .git/ (e.g., ``datalad status .git/config``)
4741 incorrectly reported the file as untracked. A workaround has been
4742 added. (`#5258 <https://github.com/datalad/datalad/issues/5258>`__)
4743
4744 - Update tests for compatibility with latest git-annex.
4745 (`#5254 <https://github.com/datalad/datalad/issues/5254>`__)
4746
4747 .. _enhancements-and-new-features-13:
4748
4749 Enhancements and new features
4750 -----------------------------
4751
4752 - `copy-file <http://datalad.readthedocs.io/en/latest/generated/man/datalad-copy-file.html>`__
4753 now aborts if .git/ is in the target directory, adding to its
4754 existing .git/ safety checks.
4755 (`#5258 <https://github.com/datalad/datalad/issues/5258>`__)
4756
4757 0.13.6 (December 14, 2020) – .
4758 ==============================
4759
4760 .. _fixes-7:
4761
4762 Fixes
4763 -----
4764
4765 - An assortment of fixes for Windows compatibility.
4766 (`#5113 <https://github.com/datalad/datalad/issues/5113>`__)
4767 (`#5119 <https://github.com/datalad/datalad/issues/5119>`__)
4768 (`#5125 <https://github.com/datalad/datalad/issues/5125>`__)
4769 (`#5127 <https://github.com/datalad/datalad/issues/5127>`__)
4770 (`#5136 <https://github.com/datalad/datalad/issues/5136>`__)
4771 (`#5201 <https://github.com/datalad/datalad/issues/5201>`__)
4772 (`#5200 <https://github.com/datalad/datalad/issues/5200>`__)
4773 (`#5214 <https://github.com/datalad/datalad/issues/5214>`__)
4774
4775 - Adding a subdataset on a system that defaults to using an adjusted
4776 branch (i.e. doesn’t support symlinks) didn’t properly set up the
4777 submodule URL if the source dataset was not in an adjusted state.
4778 (`#5127 <https://github.com/datalad/datalad/issues/5127>`__)
4779
4780 - `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__
4781 failed to push to a remote that did not have an ``annex-uuid`` value
4782 in the local ``.git/config``.
4783 (`#5148 <https://github.com/datalad/datalad/issues/5148>`__)
4784
4785 - The default renderer has been improved to avoid a spurious leading
4786 space, which led to the displayed path being incorrect in some cases.
4787 (`#5121 <https://github.com/datalad/datalad/issues/5121>`__)
4788
4789 - `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
4790 showed an uninformative error message when asked to configure an
4791 unknown remote.
4792 (`#5146 <https://github.com/datalad/datalad/issues/5146>`__)
4793
4794 - `drop <http://datalad.readthedocs.io/en/latest/generated/man/datalad-drop.html>`__
4795 confusingly relayed a suggestion from ``git annex drop`` to use
4796 ``--force``, an option that does not exist in ``datalad drop``.
4797 (`#5194 <https://github.com/datalad/datalad/issues/5194>`__)
4798
4799 - `create-sibling-github <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-github.html>`__
4800 no longer offers user/password authentication because it is no longer
4801 supported by GitHub.
4802 (`#5218 <https://github.com/datalad/datalad/issues/5218>`__)
4803
4804 - The internal command runner’s handling of the event loop has been
4805 tweaked to hopefully fix issues with running DataLad from IPython.
4806 (`#5106 <https://github.com/datalad/datalad/issues/5106>`__)
4807
4808 - SSH cleanup wasn’t reliably triggered by the ORA special remote on
4809 failure, leading to a stall with a particular version of git-annex,
4810 8.20201103. (This is also resolved on git-annex’s end as of
4811 8.20201127.)
4812 (`#5151 <https://github.com/datalad/datalad/issues/5151>`__)
4813
4814 .. _enhancements-and-new-features-14:
4815
4816 Enhancements and new features
4817 -----------------------------
4818
4819 - The credential helper no longer asks the user to repeat tokens or AWS
4820 keys. (`#5219 <https://github.com/datalad/datalad/issues/5219>`__)
4821
4822 - The new option ``datalad.locations.sockets`` controls where DataLad
4823 stores SSH sockets, allowing users to more easily work around file
4824 system and path length restrictions.
4825 (`#5238 <https://github.com/datalad/datalad/issues/5238>`__)
4826
4827 0.13.5 (October 30, 2020) – .
4828 =============================
4829
4830 .. _fixes-8:
4831
4832 Fixes
4833 -----
4834
4835 - SSH connection handling has been reworked to fix cloning on Windows.
4836 A new configuration option, ``datalad.ssh.multiplex-connections``,
4837 defaults to false on Windows.
4838 (`#5042 <https://github.com/datalad/datalad/issues/5042>`__)
4839
4840 - The ORA special remote and post-clone RIA configuration now provide
4841 authentication via DataLad’s credential mechanism and better handling
4842 of HTTP status codes.
4843 (`#5025 <https://github.com/datalad/datalad/issues/5025>`__)
4844 (`#5026 <https://github.com/datalad/datalad/issues/5026>`__)
4845
4846 - By default, if a git executable is present in the same location as
4847 git-annex, DataLad modifies ``PATH`` when running git and git-annex
4848 so that the bundled git is used. This logic has been tightened to
4849 avoid unnecessarily adjusting the path, reducing the cases where the
4850 adjustment interferes with the local environment, such as special
4851 remotes in a virtual environment being masked by the system-wide
4852 variants.
4853 (`#5035 <https://github.com/datalad/datalad/issues/5035>`__)
4854
4855 - git-annex is now consistently invoked as “git annex” rather than
4856 “git-annex” to work around failures on Windows.
4857 (`#5001 <https://github.com/datalad/datalad/issues/5001>`__)
4858
4859 - `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__
4860 called ``git annex sync ...`` on plain git repositories.
4861 (`#5051 <https://github.com/datalad/datalad/issues/5051>`__)
4862
4863 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
4864 in genernal doesn’t support registering multiple levels of untracked
4865 subdatasets, but it can now properly register nested subdatasets when
4866 all of the subdataset paths are passed explicitly (e.g.,
4867 ``datalad save -d. sub-a sub-a/sub-b``).
4868 (`#5049 <https://github.com/datalad/datalad/issues/5049>`__)
4869
4870 - When called with ``--sidecar`` and ``--explicit``,
4871 `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
4872 didn’t save the sidecar.
4873 (`#5017 <https://github.com/datalad/datalad/issues/5017>`__)
4874
4875 - A couple of spots didn’t properly quote format fields when combining
4876 substrings into a format string.
4877 (`#4957 <https://github.com/datalad/datalad/issues/4957>`__)
4878
4879 - The default credentials configured for ``indi-s3`` prevented
4880 anonymous access.
4881 (`#5045 <https://github.com/datalad/datalad/issues/5045>`__)
4882
4883 .. _enhancements-and-new-features-15:
4884
4885 Enhancements and new features
4886 -----------------------------
4887
4888 - Messages about suppressed similar results are now rate limited to
4889 improve performance when there are many similar results coming
4890 through quickly.
4891 (`#5060 <https://github.com/datalad/datalad/issues/5060>`__)
4892
4893 - `create-sibling-github <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-github.html>`__
4894 can now be told to replace an existing sibling by passing
4895 ``--existing=replace``.
4896 (`#5008 <https://github.com/datalad/datalad/issues/5008>`__)
4897
4898 - Progress bars now react to changes in the terminal’s width (requires
4899 tqdm 2.1 or later).
4900 (`#5057 <https://github.com/datalad/datalad/issues/5057>`__)
4901
4902 0.13.4 (October 6, 2020) – .
4903 ============================
4904
4905 .. _fixes-9:
4906
4907 Fixes
4908 -----
4909
4910 - Ephemeral clones mishandled bare repositories.
4911 (`#4899 <https://github.com/datalad/datalad/issues/4899>`__)
4912
4913 - The post-clone logic for configuring RIA stores didn’t consider
4914 ``https://`` URLs.
4915 (`#4977 <https://github.com/datalad/datalad/issues/4977>`__)
4916
4917 - DataLad custom remotes didn’t escape newlines in messages sent to
4918 git-annex.
4919 (`#4926 <https://github.com/datalad/datalad/issues/4926>`__)
4920
4921 - The datalad-archives special remote incorrectly treated file names as
4922 percent-encoded.
4923 (`#4953 <https://github.com/datalad/datalad/issues/4953>`__)
4924
4925 - The result handler didn’t properly escape “%” when constructing its
4926 message template.
4927 (`#4953 <https://github.com/datalad/datalad/issues/4953>`__)
4928
4929 - In v0.13.0, the tailored rendering for specific subtypes of external
4930 command failures (e.g., “out of space” or “remote not available”) was
4931 unintentionally switched to the default rendering.
4932 (`#4966 <https://github.com/datalad/datalad/issues/4966>`__)
4933
4934 - Various fixes and updates for the NDA authenticator.
4935 (`#4824 <https://github.com/datalad/datalad/issues/4824>`__)
4936
4937 - The helper for getting a versioned S3 URL did not support anonymous
4938 access or buckets with “.” in their name.
4939 (`#4985 <https://github.com/datalad/datalad/issues/4985>`__)
4940
4941 - Several issues with the handling of S3 credentials and token
4942 expiration have been addressed.
4943 (`#4927 <https://github.com/datalad/datalad/issues/4927>`__)
4944 (`#4931 <https://github.com/datalad/datalad/issues/4931>`__)
4945 (`#4952 <https://github.com/datalad/datalad/issues/4952>`__)
4946
4947 .. _enhancements-and-new-features-16:
4948
4949 Enhancements and new features
4950 -----------------------------
4951
4952 - A warning is now given if the detected Git is below v2.13.0 to let
4953 users that run into problems know that their Git version is likely
4954 the culprit.
4955 (`#4866 <https://github.com/datalad/datalad/issues/4866>`__)
4956
4957 - A fix to
4958 `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__
4959 in v0.13.2 introduced a regression that surfaces when
4960 ``push.default`` is configured to “matching” and prevents the
4961 git-annex branch from being pushed. Note that, as part of the fix,
4962 the current branch is now always pushed even when it wouldn’t be
4963 based on the configured refspec or ``push.default`` value.
4964 (`#4896 <https://github.com/datalad/datalad/issues/4896>`__)
4965
4966 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
4967
4968 - now allows spelling the empty string value of ``--since=`` as
4969 ``^`` for consistency with
4970 `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__.
4971 (`#4683 <https://github.com/datalad/datalad/issues/4683>`__)
4972 - compares a revision given to ``--since=`` with ``HEAD`` rather
4973 than the working tree to speed up the operation.
4974 (`#4448 <https://github.com/datalad/datalad/issues/4448>`__)
4975
4976 - `rerun <https://datalad.readthedocs.io/en/latest/generated/man/datalad-rerun.html>`__
4977
4978 - emits more INFO-level log messages.
4979 (`#4764 <https://github.com/datalad/datalad/issues/4764>`__)
4980 - provides better handling of adjusted branches and aborts with a
4981 clear error for cases that are not supported.
4982 (`#5328 <https://github.com/datalad/datalad/issues/5328>`__)
4983
4984 - The archives are handled with p7zip, if available, since DataLad
4985 v0.12.0. This implementation now supports .tgz and .tbz2 archives.
4986 (`#4877 <https://github.com/datalad/datalad/issues/4877>`__)
4987
4988 0.13.3 (August 28, 2020) – .
4989 ============================
4990
4991 .. _fixes-10:
4992
4993 Fixes
4994 -----
4995
4996 - Work around a Python bug that led to our asyncio-based command runner
4997 intermittently failing to capture the output of commands that exit
4998 very quickly.
4999 (`#4835 <https://github.com/datalad/datalad/issues/4835>`__)
5000
5001 - `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__
5002 displayed an overestimate of the transfer size when multiple files
5003 pointed to the same key.
5004 (`#4821 <https://github.com/datalad/datalad/issues/4821>`__)
5005
5006 - When
5007 `download-url <https://datalad.readthedocs.io/en/latest/generated/man/datalad-download-url.html>`__
5008 calls ``git annex addurl``, it catches and reports any failures
5009 rather than crashing. A change in v0.12.0 broke this handling in a
5010 particular case.
5011 (`#4817 <https://github.com/datalad/datalad/issues/4817>`__)
5012
5013 .. _enhancements-and-new-features-17:
5014
5015 Enhancements and new features
5016 -----------------------------
5017
5018 - The wrapper functions returned by decorators are now given more
5019 meaningful names to hopefully make tracebacks easier to digest.
5020 (`#4834 <https://github.com/datalad/datalad/issues/4834>`__)
5021
5022 0.13.2 (August 10, 2020) – .
5023 ============================
5024
5025 Deprecations
5026 ------------
5027
5028 - The ``allow_quick`` parameter of ``AnnexRepo.file_has_content`` and
5029 ``AnnexRepo.is_under_annex`` is now ignored and will be removed in a
5030 later release. This parameter was only relevant for git-annex
5031 versions before 7.20190912.
5032 (`#4736 <https://github.com/datalad/datalad/issues/4736>`__)
5033
5034 .. _fixes-11:
5035
5036 Fixes
5037 -----
5038
5039 - Updates for compatibility with recent git and git-annex releases.
5040 (`#4746 <https://github.com/datalad/datalad/issues/4746>`__)
5041 (`#4760 <https://github.com/datalad/datalad/issues/4760>`__)
5042 (`#4684 <https://github.com/datalad/datalad/issues/4684>`__)
5043
5044 - `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__
5045 didn’t sync the git-annex branch when ``--data=nothing`` was
5046 specified.
5047 (`#4786 <https://github.com/datalad/datalad/issues/4786>`__)
5048
5049 - The ``datalad.clone.reckless`` configuration wasn’t stored in
5050 non-annex datasets, preventing the values from being inherited by
5051 annex subdatasets.
5052 (`#4749 <https://github.com/datalad/datalad/issues/4749>`__)
5053
5054 - Running the post-update hook installed by ``create-sibling --ui``
5055 could overwrite web log files from previous runs in the unlikely
5056 event that the hook was executed multiple times in the same second.
5057 (`#4745 <https://github.com/datalad/datalad/issues/4745>`__)
5058
5059 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
5060 inspected git’s standard error in a way that could cause an attribute
5061 error. (`#4775 <https://github.com/datalad/datalad/issues/4775>`__)
5062
5063 - When cloning a repository whose ``HEAD`` points to a branch without
5064 commits,
5065 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
5066 tries to find a more useful branch to check out. It unwisely
5067 considered adjusted branches.
5068 (`#4792 <https://github.com/datalad/datalad/issues/4792>`__)
5069
5070 - Since v0.12.0, ``SSHManager.close`` hasn’t closed connections when
5071 the ``ctrl_path`` argument was explicitly given.
5072 (`#4757 <https://github.com/datalad/datalad/issues/4757>`__)
5073
5074 - When working in a dataset in which ``git annex init`` had not yet
5075 been called, the ``file_has_content`` and ``is_under_annex`` methods
5076 of ``AnnexRepo`` incorrectly took the “allow quick” code path on file
5077 systems that did not support it
5078 (`#4736 <https://github.com/datalad/datalad/issues/4736>`__)
5079
5080 Enhancements
5081 ------------
5082
5083 - `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__
5084 now assigns version 4 (random) UUIDs instead of version 1 UUIDs that
5085 encode the time and hardware address.
5086 (`#4790 <https://github.com/datalad/datalad/issues/4790>`__)
5087
5088 - The documentation for
5089 `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__
5090 now does a better job of describing the interaction between
5091 ``--dataset`` and ``PATH``.
5092 (`#4763 <https://github.com/datalad/datalad/issues/4763>`__)
5093
5094 - The ``format_commit`` and ``get_hexsha`` methods of ``GitRepo`` have
5095 been sped up.
5096 (`#4807 <https://github.com/datalad/datalad/issues/4807>`__)
5097 (`#4806 <https://github.com/datalad/datalad/issues/4806>`__)
5098
5099 - A better error message is now shown when the ``^`` or ``^.``
5100 shortcuts for ``--dataset`` do not resolve to a dataset.
5101 (`#4759 <https://github.com/datalad/datalad/issues/4759>`__)
5102
5103 - A more helpful error message is now shown if a caller tries to
5104 download an ``ftp://`` link but does not have ``request_ftp``
5105 installed.
5106 (`#4788 <https://github.com/datalad/datalad/issues/4788>`__)
5107
5108 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
5109 now tries harder to get up-to-date availability information after
5110 auto-enabling ``type=git`` special remotes.
5111 (`#2897 <https://github.com/datalad/datalad/issues/2897>`__)
5112
5113 0.13.1 (July 17, 2020) – .
5114 ==========================
5115
5116 .. _fixes-12:
5117
5118 Fixes
5119 -----
5120
5121 - Cloning a subdataset should inherit the parent’s
5122 ``datalad.clone.reckless`` value, but that did not happen when
5123 cloning via ``datalad get`` rather than ``datalad install`` or
5124 ``datalad clone``.
5125 (`#4657 <https://github.com/datalad/datalad/issues/4657>`__)
5126
5127 - The default result renderer crashed when the result did not have a
5128 ``path`` key.
5129 (`#4666 <https://github.com/datalad/datalad/issues/4666>`__)
5130 (`#4673 <https://github.com/datalad/datalad/issues/4673>`__)
5131
5132 - ``datalad push`` didn’t show information about ``git push`` errors
5133 when the output was not in the format that it expected.
5134 (`#4674 <https://github.com/datalad/datalad/issues/4674>`__)
5135
5136 - ``datalad push`` silently accepted an empty string for ``--since``
5137 even though it is an invalid value.
5138 (`#4682 <https://github.com/datalad/datalad/issues/4682>`__)
5139
5140 - Our JavaScript testing setup on Travis grew stale and has now been
5141 updated. (Thanks to Xiao Gui.)
5142 (`#4687 <https://github.com/datalad/datalad/issues/4687>`__)
5143
5144 - The new class for running Git commands (added in v0.13.0) ignored any
5145 changes to the process environment that occurred after instantiation.
5146 (`#4703 <https://github.com/datalad/datalad/issues/4703>`__)
5147
5148 .. _enhancements-and-new-features-18:
5149
5150 Enhancements and new features
5151 -----------------------------
5152
5153 - ``datalad push`` now avoids unnecessary ``git push`` dry runs and
5154 pushes all refspecs with a single ``git push`` call rather than
5155 invoking ``git push`` for each one.
5156 (`#4692 <https://github.com/datalad/datalad/issues/4692>`__)
5157 (`#4675 <https://github.com/datalad/datalad/issues/4675>`__)
5158
5159 - The readability of SSH error messages has been improved.
5160 (`#4729 <https://github.com/datalad/datalad/issues/4729>`__)
5161
5162 - ``datalad.support.annexrepo`` avoids calling
5163 ``datalad.utils.get_linux_distribution`` at import time and caches
5164 the result once it is called because, as of Python 3.8, the function
5165 uses ``distro`` underneath, adding noticeable overhead.
5166 (`#4696 <https://github.com/datalad/datalad/issues/4696>`__)
5167
5168 Third-party code should be updated to use ``get_linux_distribution``
5169 directly in the unlikely event that the code relied on the
5170 import-time call to ``get_linux_distribution`` setting the
5171 ``linux_distribution_name``, ``linux_distribution_release``, or
5172 ``on_debian_wheezy`` attributes in \`datalad.utils.
5173
5174 0.13.0 (June 23, 2020) – .
5175 ==========================
5176
5177 A handful of new commands, including ``copy-file``, ``push``, and
5178 ``create-sibling-ria``, along with various fixes and enhancements
5179
5180 .. _major-refactoring-and-deprecations-1:
5181
5182 Major refactoring and deprecations
5183 ----------------------------------
5184
5185 - The ``no_annex`` parameter of
5186 `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__,
5187 which is exposed in the Python API but not the command line, is
5188 deprecated and will be removed in a later release. Use the new
5189 ``annex`` argument instead, flipping the value. Command-line callers
5190 that use ``--no-annex`` are unaffected.
5191 (`#4321 <https://github.com/datalad/datalad/issues/4321>`__)
5192
5193 - ``datalad add``, which was deprecated in 0.12.0, has been removed.
5194 (`#4158 <https://github.com/datalad/datalad/issues/4158>`__)
5195 (`#4319 <https://github.com/datalad/datalad/issues/4319>`__)
5196
5197 - The following ``GitRepo`` and ``AnnexRepo`` methods have been
5198 removed: ``get_changed_files``, ``get_missing_files``, and
5199 ``get_deleted_files``.
5200 (`#4169 <https://github.com/datalad/datalad/issues/4169>`__)
5201 (`#4158 <https://github.com/datalad/datalad/issues/4158>`__)
5202
5203 - The ``get_branch_commits`` method of ``GitRepo`` and ``AnnexRepo``
5204 has been renamed to ``get_branch_commits_``.
5205 (`#3834 <https://github.com/datalad/datalad/issues/3834>`__)
5206
5207 - The custom ``commit`` method of ``AnnexRepo`` has been removed, and
5208 ``AnnexRepo.commit`` now resolves to the parent method,
5209 ``GitRepo.commit``.
5210 (`#4168 <https://github.com/datalad/datalad/issues/4168>`__)
5211
5212 - GitPython’s ``git.repo.base.Repo`` class is no longer available via
5213 the ``.repo`` attribute of ``GitRepo`` and ``AnnexRepo``.
5214 (`#4172 <https://github.com/datalad/datalad/issues/4172>`__)
5215
5216 - ``AnnexRepo.get_corresponding_branch`` now returns ``None`` rather
5217 than the current branch name when a managed branch is not checked
5218 out. (`#4274 <https://github.com/datalad/datalad/issues/4274>`__)
5219
5220 - The special UUID for git-annex web remotes is now available as
5221 ``datalad.consts.WEB_SPECIAL_REMOTE_UUID``. It remains accessible as
5222 ``AnnexRepo.WEB_UUID`` for compatibility, but new code should use
5223 ``consts.WEB_SPECIAL_REMOTE_UUID``
5224 (`#4460 <https://github.com/datalad/datalad/issues/4460>`__).
5225
5226 .. _fixes-13:
5227
5228 Fixes
5229 -----
5230
5231 - Widespread improvements in functionality and test coverage on Windows
5232 and crippled file systems in general.
5233 (`#4057 <https://github.com/datalad/datalad/issues/4057>`__)
5234 (`#4245 <https://github.com/datalad/datalad/issues/4245>`__)
5235 (`#4268 <https://github.com/datalad/datalad/issues/4268>`__)
5236 (`#4276 <https://github.com/datalad/datalad/issues/4276>`__)
5237 (`#4291 <https://github.com/datalad/datalad/issues/4291>`__)
5238 (`#4296 <https://github.com/datalad/datalad/issues/4296>`__)
5239 (`#4301 <https://github.com/datalad/datalad/issues/4301>`__)
5240 (`#4303 <https://github.com/datalad/datalad/issues/4303>`__)
5241 (`#4304 <https://github.com/datalad/datalad/issues/4304>`__)
5242 (`#4305 <https://github.com/datalad/datalad/issues/4305>`__)
5243 (`#4306 <https://github.com/datalad/datalad/issues/4306>`__)
5244
5245 - ``AnnexRepo.get_size_from_key`` incorrectly handled file chunks.
5246 (`#4081 <https://github.com/datalad/datalad/issues/4081>`__)
5247
5248 - `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
5249 would too readily clobber existing paths when called with
5250 ``--existing=replace``. It now gets confirmation from the user before
5251 doing so if running interactively and unconditionally aborts when
5252 running non-interactively.
5253 (`#4147 <https://github.com/datalad/datalad/issues/4147>`__)
5254
5255 - `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
5256 (`#4159 <https://github.com/datalad/datalad/issues/4159>`__)
5257
5258 - queried the incorrect branch configuration when updating non-annex
5259 repositories.
5260 - didn’t account for the fact that the local repository can be
5261 configured as the upstream “remote” for a branch.
5262
5263 - When the caller included ``--bare`` as a ``git init`` option,
5264 `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__
5265 crashed creating the bare repository, which is currently unsupported,
5266 rather than aborting with an informative error message.
5267 (`#4065 <https://github.com/datalad/datalad/issues/4065>`__)
5268
5269 - The logic for automatically propagating the ‘origin’ remote when
5270 cloning a local source could unintentionally trigger a fetch of a
5271 non-local remote.
5272 (`#4196 <https://github.com/datalad/datalad/issues/4196>`__)
5273
5274 - All remaining ``get_submodules()`` call sites that relied on the
5275 temporary compatibility layer added in v0.12.0 have been updated.
5276 (`#4348 <https://github.com/datalad/datalad/issues/4348>`__)
5277
5278 - The custom result summary renderer for
5279 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__,
5280 which was visible with ``--output-format=tailored``, displayed
5281 incorrect and confusing information in some cases. The custom
5282 renderer has been removed entirely.
5283 (`#4471 <https://github.com/datalad/datalad/issues/4471>`__)
5284
5285 - The documentation for the Python interface of a command listed an
5286 incorrect default when the command overrode the value of command
5287 parameters such as ``result_renderer``.
5288 (`#4480 <https://github.com/datalad/datalad/issues/4480>`__)
5289
5290 .. _enhancements-and-new-features-19:
5291
5292 Enhancements and new features
5293 -----------------------------
5294
5295 - The default result renderer learned to elide a chain of results after
5296 seeing ten consecutive results that it considers similar, which
5297 improves the display of actions that have many results (e.g., saving
5298 hundreds of files).
5299 (`#4337 <https://github.com/datalad/datalad/issues/4337>`__)
5300
5301 - The default result renderer, in addition to “tailored” result
5302 renderer, now triggers the custom summary renderer, if any.
5303 (`#4338 <https://github.com/datalad/datalad/issues/4338>`__)
5304
5305 - The new command
5306 `create-sibling-ria <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-ria.html>`__
5307 provides support for creating a sibling in a `RIA
5308 store <http://handbook.datalad.org/en/latest/usecases/datastorage_for_institutions.html>`__.
5309 (`#4124 <https://github.com/datalad/datalad/issues/4124>`__)
5310
5311 - DataLad ships with a new special remote, git-annex-remote-ora, for
5312 interacting with `RIA
5313 stores <http://handbook.datalad.org/en/latest/usecases/datastorage_for_institutions.html>`__
5314 and a new command
5315 `export-archive-ora <http://datalad.readthedocs.io/en/latest/generated/man/datalad-export-archive-ora.html>`__
5316 for exporting an archive from a local annex object store.
5317 (`#4260 <https://github.com/datalad/datalad/issues/4260>`__)
5318 (`#4203 <https://github.com/datalad/datalad/issues/4203>`__)
5319
5320 - The new command
5321 `push <http://datalad.readthedocs.io/en/latest/generated/man/datalad-push.html>`__
5322 provides an alternative interface to
5323 `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
5324 for pushing a dataset hierarchy to a sibling.
5325 (`#4206 <https://github.com/datalad/datalad/issues/4206>`__)
5326 (`#4581 <https://github.com/datalad/datalad/issues/4581>`__)
5327 (`#4617 <https://github.com/datalad/datalad/issues/4617>`__)
5328 (`#4620 <https://github.com/datalad/datalad/issues/4620>`__)
5329
5330 - The new command
5331 `copy-file <http://datalad.readthedocs.io/en/latest/generated/man/datalad-copy-file.html>`__
5332 copies files and associated availability information from one dataset
5333 to another.
5334 (`#4430 <https://github.com/datalad/datalad/issues/4430>`__)
5335
5336 - The command examples have been expanded and improved.
5337 (`#4091 <https://github.com/datalad/datalad/issues/4091>`__)
5338 (`#4314 <https://github.com/datalad/datalad/issues/4314>`__)
5339 (`#4464 <https://github.com/datalad/datalad/issues/4464>`__)
5340
5341 - The tooling for linking to the `DataLad
5342 Handbook <http://handbook.datalad.org>`__ from DataLad’s
5343 documentation has been improved.
5344 (`#4046 <https://github.com/datalad/datalad/issues/4046>`__)
5345
5346 - The ``--reckless`` parameter of
5347 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
5348 and
5349 `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
5350 learned two new modes:
5351
5352 - “ephemeral”, where the .git/annex/ of the cloned repository is
5353 symlinked to the local source repository’s.
5354 (`#4099 <https://github.com/datalad/datalad/issues/4099>`__)
5355 - “shared-{group|all\|…}” that can be used to set up datasets for
5356 collaborative write access.
5357 (`#4324 <https://github.com/datalad/datalad/issues/4324>`__)
5358
5359 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
5360
5361 - learned to handle dataset aliases in RIA stores when given a URL
5362 of the form ``ria+<protocol>://<storelocation>#~<aliasname>``.
5363 (`#4459 <https://github.com/datalad/datalad/issues/4459>`__)
5364 - now checks ``datalad.get.subdataset-source-candidate-NAME`` to see
5365 if ``NAME`` starts with three digits, which is taken as a “cost”.
5366 Sources with lower costs will be tried first.
5367 (`#4619 <https://github.com/datalad/datalad/issues/4619>`__)
5368
5369 - `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
5370 (`#4167 <https://github.com/datalad/datalad/issues/4167>`__)
5371
5372 - learned to disallow non-fast-forward updates when ``ff-only`` is
5373 given to the ``--merge`` option.
5374 - gained a ``--follow`` option that controls how ``--merge``
5375 behaves, adding support for merging in the revision that is
5376 registered in the parent dataset rather than merging in the
5377 configured branch from the sibling.
5378 - now provides a result record for merge events.
5379
5380 - `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
5381 now supports local paths as targets in addition to SSH URLs.
5382 (`#4187 <https://github.com/datalad/datalad/issues/4187>`__)
5383
5384 - `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
5385 now
5386
5387 - shows a warning if the caller requests to delete a sibling that
5388 does not exist.
5389 (`#4257 <https://github.com/datalad/datalad/issues/4257>`__)
5390 - phrases its warning about non-annex repositories in a less
5391 alarming way.
5392 (`#4323 <https://github.com/datalad/datalad/issues/4323>`__)
5393
5394 - The rendering of command errors has been improved.
5395 (`#4157 <https://github.com/datalad/datalad/issues/4157>`__)
5396
5397 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
5398 now
5399
5400 - displays a message to signal that the working tree is clean,
5401 making it more obvious that no results being rendered corresponds
5402 to a clean state.
5403 (`#4106 <https://github.com/datalad/datalad/issues/4106>`__)
5404 - provides a stronger warning against using ``--to-git``.
5405 (`#4290 <https://github.com/datalad/datalad/issues/4290>`__)
5406
5407 - `diff <http://datalad.readthedocs.io/en/latest/generated/man/datalad-diff.html>`__
5408 and
5409 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
5410 learned about scenarios where they could avoid unnecessary and
5411 expensive work.
5412 (`#4526 <https://github.com/datalad/datalad/issues/4526>`__)
5413 (`#4544 <https://github.com/datalad/datalad/issues/4544>`__)
5414 (`#4549 <https://github.com/datalad/datalad/issues/4549>`__)
5415
5416 - Calling
5417 `diff <http://datalad.readthedocs.io/en/latest/generated/man/datalad-diff.html>`__
5418 without ``--recursive`` but with a path constraint within a
5419 subdataset (“/”) now traverses into the subdataset, as “/” would,
5420 restricting its report to “/”.
5421 (`#4235 <https://github.com/datalad/datalad/issues/4235>`__)
5422
5423 - New option ``datalad.annex.retry`` controls how many times git-annex
5424 will retry on a failed transfer. It defaults to 3 and can be set to 0
5425 to restore the previous behavior.
5426 (`#4382 <https://github.com/datalad/datalad/issues/4382>`__)
5427
5428 - `wtf <http://datalad.readthedocs.io/en/latest/generated/man/datalad-wtf.html>`__
5429 now warns when the specified dataset does not exist.
5430 (`#4331 <https://github.com/datalad/datalad/issues/4331>`__)
5431
5432 - The ``repr`` and ``str`` output of the dataset and repo classes got a
5433 facelift.
5434 (`#4420 <https://github.com/datalad/datalad/issues/4420>`__)
5435 (`#4435 <https://github.com/datalad/datalad/issues/4435>`__)
5436 (`#4439 <https://github.com/datalad/datalad/issues/4439>`__)
5437
5438 - The DataLad Singularity container now comes with p7zip-full.
5439
5440 - DataLad emits a log message when the current working directory is
5441 resolved to a different location due to a symlink. This is now logged
5442 at the DEBUG rather than WARNING level, as it typically does not
5443 indicate a problem.
5444 (`#4426 <https://github.com/datalad/datalad/issues/4426>`__)
5445
5446 - DataLad now lets the caller know that ``git annex init`` is scanning
5447 for unlocked files, as this operation can be slow in some
5448 repositories.
5449 (`#4316 <https://github.com/datalad/datalad/issues/4316>`__)
5450
5451 - The ``log_progress`` helper learned how to set the starting point to
5452 a non-zero value and how to update the total of an existing progress
5453 bar, two features needed for planned improvements to how some
5454 commands display their progress.
5455 (`#4438 <https://github.com/datalad/datalad/issues/4438>`__)
5456
5457 - The ``ExternalVersions`` object, which is used to check versions of
5458 Python modules and external tools (e.g., git-annex), gained an
5459 ``add`` method that enables DataLad extensions and other third-party
5460 code to include other programs of interest.
5461 (`#4441 <https://github.com/datalad/datalad/issues/4441>`__)
5462
5463 - All of the remaining spots that use GitPython have been rewritten
5464 without it. Most notably, this includes rewrites of the ``clone``,
5465 ``fetch``, and ``push`` methods of ``GitRepo``.
5466 (`#4080 <https://github.com/datalad/datalad/issues/4080>`__)
5467 (`#4087 <https://github.com/datalad/datalad/issues/4087>`__)
5468 (`#4170 <https://github.com/datalad/datalad/issues/4170>`__)
5469 (`#4171 <https://github.com/datalad/datalad/issues/4171>`__)
5470 (`#4175 <https://github.com/datalad/datalad/issues/4175>`__)
5471 (`#4172 <https://github.com/datalad/datalad/issues/4172>`__)
5472
5473 - When ``GitRepo.commit`` splits its operation across multiple calls to
5474 avoid exceeding the maximum command line length, it now amends to
5475 initial commit rather than creating multiple commits.
5476 (`#4156 <https://github.com/datalad/datalad/issues/4156>`__)
5477
5478 - ``GitRepo`` gained a ``get_corresponding_branch`` method (which
5479 always returns None), allowing a caller to invoke the method without
5480 needing to check if the underlying repo class is ``GitRepo`` or
5481 ``AnnexRepo``.
5482 (`#4274 <https://github.com/datalad/datalad/issues/4274>`__)
5483
5484 - A new helper function ``datalad.core.local.repo.repo_from_path``
5485 returns a repo class for a specified path.
5486 (`#4273 <https://github.com/datalad/datalad/issues/4273>`__)
5487
5488 - New ``AnnexRepo`` method ``localsync`` performs a ``git annex sync``
5489 that disables external interaction and is particularly useful for
5490 propagating changes on an adjusted branch back to the main branch.
5491 (`#4243 <https://github.com/datalad/datalad/issues/4243>`__)
5492
5493 0.12.7 (May 22, 2020) – .
5494 =========================
5495
5496 .. _fixes-14:
5497
5498 Fixes
5499 -----
5500
5501 - Requesting tailored output (``--output=tailored``) from a command
5502 with a custom result summary renderer produced repeated output.
5503 (`#4463 <https://github.com/datalad/datalad/issues/4463>`__)
5504
5505 - A longstanding regression in argcomplete-based command-line
5506 completion for Bash has been fixed. You can enable completion by
5507 configuring a Bash startup file to run
5508 ``eval "$(register-python-argcomplete datalad)"`` or source
5509 DataLad’s ``tools/cmdline-completion``. The latter should work for
5510 Zsh as well.
5511 (`#4477 <https://github.com/datalad/datalad/issues/4477>`__)
5512
5513 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
5514 didn’t prevent ``git-fetch`` from recursing into submodules, leading
5515 to a failure when the registered submodule was not present locally
5516 and the submodule did not have a remote named ‘origin’.
5517 (`#4560 <https://github.com/datalad/datalad/issues/4560>`__)
5518
5519 - `addurls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-addurls.html>`__
5520 botched path handling when the file name format started with “./” and
5521 the call was made from a subdirectory of the dataset.
5522 (`#4504 <https://github.com/datalad/datalad/issues/4504>`__)
5523
5524 - Double dash options in manpages were unintentionally escaped.
5525 (`#4332 <https://github.com/datalad/datalad/issues/4332>`__)
5526
5527 - The check for HTTP authentication failures crashed in situations
5528 where content came in as bytes rather than unicode.
5529 (`#4543 <https://github.com/datalad/datalad/issues/4543>`__)
5530
5531 - A check in ``AnnexRepo.whereis`` could lead to a type error.
5532 (`#4552 <https://github.com/datalad/datalad/issues/4552>`__)
5533
5534 - When installing a dataset to obtain a subdataset,
5535 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
5536 confusingly displayed a message that described the containing dataset
5537 as “underneath” the subdataset.
5538 (`#4456 <https://github.com/datalad/datalad/issues/4456>`__)
5539
5540 - A couple of Makefile rules didn’t properly quote paths.
5541 (`#4481 <https://github.com/datalad/datalad/issues/4481>`__)
5542
5543 - With DueCredit support enabled (``DUECREDIT_ENABLE=1``), the query
5544 for metadata information could flood the output with warnings if
5545 datasets didn’t have aggregated metadata. The warnings are now
5546 silenced, with the overall failure of a
5547 `metadata <http://datalad.readthedocs.io/en/latest/generated/man/datalad-metadata.html>`__
5548 call logged at the debug level.
5549 (`#4568 <https://github.com/datalad/datalad/issues/4568>`__)
5550
5551 .. _enhancements-and-new-features-20:
5552
5553 Enhancements and new features
5554 -----------------------------
5555
5556 - The resource identifier helper learned to recognize URLs with
5557 embedded Git transport information, such as
5558 gcrypt::https://example.com.
5559 (`#4529 <https://github.com/datalad/datalad/issues/4529>`__)
5560
5561 - When running non-interactively, a more informative error is now
5562 signaled when the UI backend, which cannot display a question, is
5563 asked to do so.
5564 (`#4553 <https://github.com/datalad/datalad/issues/4553>`__)
5565
5566 0.12.6 (April 23, 2020) – .
5567 ===========================
5568
5569 .. _major-refactoring-and-deprecations-2:
5570
5571 Major refactoring and deprecations
5572 ----------------------------------
5573
5574 - The value of ``datalad.support.annexrep.N_AUTO_JOBS`` is no longer
5575 considered. The variable will be removed in a later release.
5576 (`#4409 <https://github.com/datalad/datalad/issues/4409>`__)
5577
5578 .. _fixes-15:
5579
5580 Fixes
5581 -----
5582
5583 - Staring with v0.12.0, ``datalad save`` recorded the current branch of
5584 a parent dataset as the ``branch`` value in the .gitmodules entry for
5585 a subdataset. This behavior is problematic for a few reasons and has
5586 been reverted.
5587 (`#4375 <https://github.com/datalad/datalad/issues/4375>`__)
5588
5589 - The default for the ``--jobs`` option, “auto”, instructed DataLad to
5590 pass a value to git-annex’s ``--jobs`` equal to
5591 ``min(8, max(3, <number of CPUs>))``, which could lead to issues
5592 due to the large number of child processes spawned and file
5593 descriptors opened. To avoid this behavior, ``--jobs=auto`` now
5594 results in git-annex being called with ``--jobs=1`` by default.
5595 Configure the new option ``datalad.runtime.max-annex-jobs`` to
5596 control the maximum value that will be considered when
5597 ``--jobs='auto'``.
5598 (`#4409 <https://github.com/datalad/datalad/issues/4409>`__)
5599
5600 - Various commands have been adjusted to better handle the case where a
5601 remote’s HEAD ref points to an unborn branch.
5602 (`#4370 <https://github.com/datalad/datalad/issues/4370>`__)
5603
5604 - `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
5605
5606 - learned to use the query as a regular expression that restricts
5607 the keys that are shown for ``--show-keys short``.
5608 (`#4354 <https://github.com/datalad/datalad/issues/4354>`__)
5609 - gives a more helpful message when query is an invalid regular
5610 expression.
5611 (`#4398 <https://github.com/datalad/datalad/issues/4398>`__)
5612
5613 - The code for parsing Git configuration did not follow Git’s behavior
5614 of accepting a key with no value as shorthand for key=true.
5615 (`#4421 <https://github.com/datalad/datalad/issues/4421>`__)
5616
5617 - ``AnnexRepo.info`` needed a compatibility update for a change in how
5618 git-annex reports file names.
5619 (`#4431 <https://github.com/datalad/datalad/issues/4431>`__)
5620
5621 - `create-sibling-github <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-github.html>`__
5622 did not gracefully handle a token that did not have the necessary
5623 permissions.
5624 (`#4400 <https://github.com/datalad/datalad/issues/4400>`__)
5625
5626 .. _enhancements-and-new-features-21:
5627
5628 Enhancements and new features
5629 -----------------------------
5630
5631 - `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
5632 learned to use the query as a regular expression that restricts the
5633 keys that are shown for ``--show-keys short``.
5634 (`#4354 <https://github.com/datalad/datalad/issues/4354>`__)
5635
5636 - ``datalad <subcommand>`` learned to point to the
5637 `datalad-container <https://github.com/datalad/datalad-container>`__
5638 extension when a subcommand from that extension is given but the
5639 extension is not installed.
5640 (`#4400 <https://github.com/datalad/datalad/issues/4400>`__)
5641 (`#4174 <https://github.com/datalad/datalad/issues/4174>`__)
5642
5643 0.12.5 (Apr 02, 2020) – a small step for datalad …
5644 ==================================================
5645
5646 Fix some bugs and make the world an even better place.
5647
5648 .. _fixes-16:
5649
5650 Fixes
5651 -----
5652
5653 - Our ``log_progress`` helper mishandled the initial display and step
5654 of the progress bar.
5655 (`#4326 <https://github.com/datalad/datalad/issues/4326>`__)
5656
5657 - ``AnnexRepo.get_content_annexinfo`` is designed to accept
5658 ``init=None``, but passing that led to an error.
5659 (`#4330 <https://github.com/datalad/datalad/issues/4330>`__)
5660
5661 - Update a regular expression to handle an output change in Git
5662 v2.26.0. (`#4328 <https://github.com/datalad/datalad/issues/4328>`__)
5663
5664 - We now set ``LC_MESSAGES`` to ‘C’ while running git to avoid failures
5665 when parsing output that is marked for translation.
5666 (`#4342 <https://github.com/datalad/datalad/issues/4342>`__)
5667
5668 - The helper for decoding JSON streams loaded the last line of input
5669 without decoding it if the line didn’t end with a new line, a
5670 regression introduced in the 0.12.0 release.
5671 (`#4361 <https://github.com/datalad/datalad/issues/4361>`__)
5672
5673 - The clone command failed to git-annex-init a fresh clone whenever it
5674 considered to add the origin of the origin as a remote.
5675 (`#4367 <https://github.com/datalad/datalad/issues/4367>`__)
5676
5677 0.12.4 (Mar 19, 2020) – Windows?!
5678 =================================
5679
5680 The main purpose of this release is to have one on PyPi that has no
5681 associated wheel to enable a working installation on Windows
5682 (`#4315 <https://github.com/datalad/datalad/issues/4315>`__).
5683
5684 .. _fixes-17:
5685
5686 Fixes
5687 -----
5688
5689 - The description of the ``log.outputs`` config switch did not keep up
5690 with code changes and incorrectly stated that the output would be
5691 logged at the DEBUG level; logging actually happens at a lower level.
5692 (`#4317 <https://github.com/datalad/datalad/issues/4317>`__)
5693
5694 0.12.3 (March 16, 2020) – .
5695 ===========================
5696
5697 Updates for compatibility with the latest git-annex, along with a few
5698 miscellaneous fixes
5699
5700 .. _major-refactoring-and-deprecations-3:
5701
5702 Major refactoring and deprecations
5703 ----------------------------------
5704
5705 - All spots that raised a ``NoDatasetArgumentFound`` exception now
5706 raise a ``NoDatasetFound`` exception to better reflect the situation:
5707 it is the *dataset* rather than the *argument* that is not found. For
5708 compatibility, the latter inherits from the former, but new code
5709 should prefer the latter.
5710 (`#4285 <https://github.com/datalad/datalad/issues/4285>`__)
5711
5712 .. _fixes-18:
5713
5714 Fixes
5715 -----
5716
5717 - Updates for compatibility with git-annex version 8.20200226.
5718 (`#4214 <https://github.com/datalad/datalad/issues/4214>`__)
5719
5720 - ``datalad export-to-figshare`` failed to export if the generated
5721 title was fewer than three characters. It now queries the caller for
5722 the title and guards against titles that are too short.
5723 (`#4140 <https://github.com/datalad/datalad/issues/4140>`__)
5724
5725 - Authentication was requested multiple times when git-annex launched
5726 parallel downloads from the ``datalad`` special remote.
5727 (`#4308 <https://github.com/datalad/datalad/issues/4308>`__)
5728
5729 - At verbose logging levels, DataLad requests that git-annex display
5730 debugging information too. Work around a bug in git-annex that
5731 prevented that from happening.
5732 (`#4212 <https://github.com/datalad/datalad/issues/4212>`__)
5733
5734 - The internal command runner looked in the wrong place for some
5735 configuration variables, including ``datalad.log.outputs``, resulting
5736 in the default value always being used.
5737 (`#4194 <https://github.com/datalad/datalad/issues/4194>`__)
5738
5739 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
5740 failed when trying to publish to a git-lfs special remote for the
5741 first time.
5742 (`#4200 <https://github.com/datalad/datalad/issues/4200>`__)
5743
5744 - ``AnnexRepo.set_remote_url`` is supposed to establish shared SSH
5745 connections but failed to do so.
5746 (`#4262 <https://github.com/datalad/datalad/issues/4262>`__)
5747
5748 .. _enhancements-and-new-features-22:
5749
5750 Enhancements and new features
5751 -----------------------------
5752
5753 - The message provided when a command cannot determine what dataset to
5754 operate on has been improved.
5755 (`#4285 <https://github.com/datalad/datalad/issues/4285>`__)
5756
5757 - The “aws-s3” authentication type now allows specifying the host
5758 through “aws-s3_host”, which was needed to work around an
5759 authorization error due to a longstanding upstream bug.
5760 (`#4239 <https://github.com/datalad/datalad/issues/4239>`__)
5761
5762 - The xmp metadata extractor now recognizes “.wav” files.
5763
5764 0.12.2 (Jan 28, 2020) – Smoothen the ride
5765 =========================================
5766
5767 Mostly a bugfix release with various robustifications, but also makes
5768 the first step towards versioned dataset installation requests.
5769
5770 .. _major-refactoring-and-deprecations-4:
5771
5772 Major refactoring and deprecations
5773 ----------------------------------
5774
5775 - The minimum required version for GitPython is now 2.1.12.
5776 (`#4070 <https://github.com/datalad/datalad/issues/4070>`__)
5777
5778 .. _fixes-19:
5779
5780 Fixes
5781 -----
5782
5783 - The class for handling configuration values, ``ConfigManager``,
5784 inappropriately considered the current working directory’s dataset,
5785 if any, for both reading and writing when instantiated with
5786 ``dataset=None``. This misbehavior is fairly inaccessible through
5787 typical use of DataLad. It affects ``datalad.cfg``, the top-level
5788 configuration instance that should not consider repository-specific
5789 values. It also affects Python users that call ``Dataset`` with a
5790 path that does not yet exist and persists until that dataset is
5791 created. (`#4078 <https://github.com/datalad/datalad/issues/4078>`__)
5792
5793 - `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
5794 saved the dataset when called with ``--merge``, which is unnecessary
5795 and risks committing unrelated changes.
5796 (`#3996 <https://github.com/datalad/datalad/issues/3996>`__)
5797
5798 - Confusing and irrelevant information about Python defaults have been
5799 dropped from the command-line help.
5800 (`#4002 <https://github.com/datalad/datalad/issues/4002>`__)
5801
5802 - The logic for automatically propagating the ‘origin’ remote when
5803 cloning a local source didn’t properly account for relative paths.
5804 (`#4045 <https://github.com/datalad/datalad/issues/4045>`__)
5805
5806 - Various fixes to file name handling and quoting on Windows.
5807 (`#4049 <https://github.com/datalad/datalad/issues/4049>`__)
5808 (`#4050 <https://github.com/datalad/datalad/issues/4050>`__)
5809
5810 - When cloning failed, error lines were not bubbled up to the user in
5811 some scenarios.
5812 (`#4060 <https://github.com/datalad/datalad/issues/4060>`__)
5813
5814 .. _enhancements-and-new-features-23:
5815
5816 Enhancements and new features
5817 -----------------------------
5818
5819 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
5820 (and thus
5821 `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__)
5822
5823 - now propagates the ``reckless`` mode from the superdataset when
5824 cloning a dataset into it.
5825 (`#4037 <https://github.com/datalad/datalad/issues/4037>`__)
5826 - gained support for ``ria+<protocol>://`` URLs that point to
5827 `RIA <http://handbook.datalad.org/en/latest/usecases/datastorage_for_institutions.html>`__
5828 stores.
5829 (`#4022 <https://github.com/datalad/datalad/issues/4022>`__)
5830 - learned to read “@version” from ``ria+`` URLs and install that
5831 version of a dataset
5832 (`#4036 <https://github.com/datalad/datalad/issues/4036>`__) and
5833 to apply URL rewrites configured through Git’s ``url.*.insteadOf``
5834 mechanism
5835 (`#4064 <https://github.com/datalad/datalad/issues/4064>`__).
5836 - now copies ``datalad.get.subdataset-source-candidate-<name>``
5837 options configured within the superdataset into the subdataset.
5838 This is particularly useful for RIA data stores.
5839 (`#4073 <https://github.com/datalad/datalad/issues/4073>`__)
5840
5841 - Archives are now (optionally) handled with 7-Zip instead of
5842 ``patool``. 7-Zip will be used by default, but ``patool`` will be
5843 used on non-Windows systems if the ``datalad.runtime.use-patool``
5844 option is set or the ``7z`` executable is not found.
5845 (`#4041 <https://github.com/datalad/datalad/issues/4041>`__)
5846
5847 0.12.1 (Jan 15, 2020) – Small bump after big bang
5848 =================================================
5849
5850 Fix some fallout after major release.
5851
5852 .. _fixes-20:
5853
5854 Fixes
5855 -----
5856
5857 - Revert incorrect relative path adjustment to URLs in
5858 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__.
5859 (`#3538 <https://github.com/datalad/datalad/issues/3538>`__)
5860
5861 - Various small fixes to internal helpers and test to run on Windows
5862 (`#2566 <https://github.com/datalad/datalad/issues/2566>`__)
5863 (`#2534 <https://github.com/datalad/datalad/issues/2534>`__)
5864
5865 0.12.0 (Jan 11, 2020) – Krakatoa
5866 ================================
5867
5868 This release is the result of more than a year of development that
5869 includes fixes for a large number of issues, yielding more robust
5870 behavior across a wider range of use cases, and introduces major changes
5871 in API and behavior. It is the first release for which extensive user
5872 documentation is available in a dedicated `DataLad
5873 Handbook <http://handbook.datalad.org>`__. Python 3 (3.5 and later) is
5874 now the only supported Python flavor.
5875
5876 Major changes 0.12 vs 0.11
5877 --------------------------
5878
5879 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
5880 fully replaces
5881 `add <http://datalad.readthedocs.io/en/latest/generated/man/datalad-add.html>`__
5882 (which is obsolete now, and will be removed in a future release).
5883
5884 - A new Git-annex aware
5885 `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__
5886 command enables detailed inspection of dataset hierarchies. The
5887 previously available
5888 `diff <http://datalad.readthedocs.io/en/latest/generated/man/datalad-diff.html>`__
5889 command has been adjusted to match
5890 `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__
5891 in argument semantics and behavior.
5892
5893 - The ability to configure dataset procedures prior and after the
5894 execution of particular commands has been replaced by a flexible
5895 “hook” mechanism that is able to run arbitrary DataLad commands
5896 whenever command results are detected that match a specification.
5897
5898 - Support of the Windows platform has been improved substantially.
5899 While performance and feature coverage on Windows still falls behind
5900 Unix-like systems, typical data consumer use cases, and standard
5901 dataset operations, such as
5902 `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__
5903 and
5904 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__,
5905 are now working. Basic support for data provenance capture via
5906 `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
5907 is also functional.
5908
5909 - Support for Git-annex direct mode repositories has been removed,
5910 following the end of support in Git-annex itself.
5911
5912 - The semantics of relative paths in command line arguments have
5913 changed. Previously, a call
5914 ``datalad save --dataset /tmp/myds some/relpath`` would have been
5915 interpreted as saving a file at ``/tmp/myds/some/relpath`` into
5916 dataset ``/tmp/myds``. This has changed to saving
5917 ``$PWD/some/relpath`` into dataset ``/tmp/myds``. More generally,
5918 relative paths are now always treated as relative to the current
5919 working directory, except for path arguments of
5920 `Dataset <http://docs.datalad.org/en/latest/generated/datalad.api.Dataset.html>`__
5921 class instance methods of the Python API. The resulting partial
5922 duplication of path specifications between path and dataset arguments
5923 is mitigated by the introduction of two special symbols that can be
5924 given as dataset argument: ``^`` and ``^.``, which identify the
5925 topmost superdataset and the closest dataset that contains the
5926 working directory, respectively.
5927
5928 - The concept of a “core API” has been introduced. Commands situated in
5929 the module ``datalad.core`` (such as
5930 `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__,
5931 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__,
5932 `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__,
5933 `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__,
5934 `diff <http://datalad.readthedocs.io/en/latest/generated/man/datalad-diff.html>`__)
5935 receive additional scrutiny regarding API and implementation, and are
5936 meant to provide longer-term stability. Application developers are
5937 encouraged to preferentially build on these commands.
5938
5939 Major refactoring and deprecations since 0.12.0rc6
5940 --------------------------------------------------
5941
5942 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
5943 has been incorporated into the growing core API. The public
5944 ``--alternative-source`` parameter has been removed, and a
5945 ``clone_dataset`` function with multi-source capabilities is provided
5946 instead. The ``--reckless`` parameter can now take literal mode
5947 labels instead of just being a binary flag, but backwards
5948 compatibility is maintained.
5949
5950 - The ``get_file_content`` method of ``GitRepo`` was no longer used
5951 internally or in any known DataLad extensions and has been removed.
5952 (`#3812 <https://github.com/datalad/datalad/issues/3812>`__)
5953
5954 - The function ``get_dataset_root`` has been replaced by
5955 ``rev_get_dataset_root``. ``rev_get_dataset_root`` remains as a
5956 compatibility alias and will be removed in a later release.
5957 (`#3815 <https://github.com/datalad/datalad/issues/3815>`__)
5958
5959 - The ``add_sibling`` module, marked obsolete in v0.6.0, has been
5960 removed. (`#3871 <https://github.com/datalad/datalad/issues/3871>`__)
5961
5962 - ``mock`` is no longer declared as an external dependency because we
5963 can rely on it being in the standard library now that our minimum
5964 required Python version is 3.5.
5965 (`#3860 <https://github.com/datalad/datalad/issues/3860>`__)
5966
5967 - `download-url <https://datalad.readthedocs.io/en/latest/generated/man/datalad-download-url.html>`__
5968 now requires that directories be indicated with a trailing slash
5969 rather than interpreting a path as directory when it doesn’t exist.
5970 This avoids confusion that can result from typos and makes it
5971 possible to support directory targets that do not exist.
5972 (`#3854 <https://github.com/datalad/datalad/issues/3854>`__)
5973
5974 - The ``dataset_only`` argument of the ``ConfigManager`` class is
5975 deprecated. Use ``source="dataset"`` instead.
5976 (`#3907 <https://github.com/datalad/datalad/issues/3907>`__)
5977
5978 - The ``--proc-pre`` and ``--proc-post`` options have been removed, and
5979 configuration values for ``datalad.COMMAND.proc-pre`` and
5980 ``datalad.COMMAND.proc-post`` are no longer honored. The new result
5981 hook mechanism provides an alternative for ``proc-post`` procedures.
5982 (`#3963 <https://github.com/datalad/datalad/issues/3963>`__)
5983
5984 Fixes since 0.12.0rc6
5985 ---------------------
5986
5987 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
5988 crashed when called with a detached HEAD. It now aborts with an
5989 informative message.
5990 (`#3804 <https://github.com/datalad/datalad/issues/3804>`__)
5991
5992 - Since 0.12.0rc6 the call to
5993 `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
5994 in
5995 `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
5996 resulted in a spurious warning.
5997 (`#3877 <https://github.com/datalad/datalad/issues/3877>`__)
5998
5999 - `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
6000 crashed if it encountered an annex repository that was marked as
6001 dead. (`#3892 <https://github.com/datalad/datalad/issues/3892>`__)
6002
6003 - The update of
6004 `rerun <https://datalad.readthedocs.io/en/latest/generated/man/datalad-rerun.html>`__
6005 in v0.12.0rc3 for the rewritten
6006 `diff <http://datalad.readthedocs.io/en/latest/generated/man/datalad-diff.html>`__
6007 command didn’t account for a change in the output of ``diff``,
6008 leading to ``rerun --report`` unintentionally including unchanged
6009 files in its diff values.
6010 (`#3873 <https://github.com/datalad/datalad/issues/3873>`__)
6011
6012 - In 0.12.0rc5
6013 `download-url <https://datalad.readthedocs.io/en/latest/generated/man/datalad-download-url.html>`__
6014 was updated to follow the new path handling logic, but its calls to
6015 AnnexRepo weren’t properly adjusted, resulting in incorrect path
6016 handling when the called from a dataset subdirectory.
6017 (`#3850 <https://github.com/datalad/datalad/issues/3850>`__)
6018
6019 - `download-url <https://datalad.readthedocs.io/en/latest/generated/man/datalad-download-url.html>`__
6020 called ``git annex addurl`` in a way that failed to register a URL
6021 when its header didn’t report the content size.
6022 (`#3911 <https://github.com/datalad/datalad/issues/3911>`__)
6023
6024 - With Git v2.24.0, saving new subdatasets failed due to a bug in that
6025 Git release.
6026 (`#3904 <https://github.com/datalad/datalad/issues/3904>`__)
6027
6028 - With DataLad configured to stop on failure (e.g., specifying
6029 ``--on-failure=stop`` from the command line), a failing result record
6030 was not rendered.
6031 (`#3863 <https://github.com/datalad/datalad/issues/3863>`__)
6032
6033 - Installing a subdataset yielded an “ok” status in cases where the
6034 repository was not yet in its final state, making it ineffective for
6035 a caller to operate on the repository in response to the result.
6036 (`#3906 <https://github.com/datalad/datalad/issues/3906>`__)
6037
6038 - The internal helper for converting git-annex’s JSON output did not
6039 relay information from the “error-messages” field.
6040 (`#3931 <https://github.com/datalad/datalad/issues/3931>`__)
6041
6042 - `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
6043 reported relative paths that were confusingly not relative to the
6044 current directory in some cases. It now always reports absolute
6045 paths. (`#3959 <https://github.com/datalad/datalad/issues/3959>`__)
6046
6047 - `diff <http://datalad.readthedocs.io/en/latest/generated/man/datalad-diff.html>`__
6048 inappropriately reported files as deleted in some cases when ``to``
6049 was a value other than ``None``.
6050 (`#3999 <https://github.com/datalad/datalad/issues/3999>`__)
6051
6052 - An assortment of fixes for Windows compatibility.
6053 (`#3971 <https://github.com/datalad/datalad/issues/3971>`__)
6054 (`#3974 <https://github.com/datalad/datalad/issues/3974>`__)
6055 (`#3975 <https://github.com/datalad/datalad/issues/3975>`__)
6056 (`#3976 <https://github.com/datalad/datalad/issues/3976>`__)
6057 (`#3979 <https://github.com/datalad/datalad/issues/3979>`__)
6058
6059 - Subdatasets installed from a source given by relative path will now
6060 have this relative path used as ‘url’ in their .gitmodules record,
6061 instead of an absolute path generated by Git.
6062 (`#3538 <https://github.com/datalad/datalad/issues/3538>`__)
6063
6064 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
6065 will now correctly interpret ‘~/…’ paths as absolute path
6066 specifications.
6067 (`#3958 <https://github.com/datalad/datalad/issues/3958>`__)
6068
6069 - `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
6070 mistakenly reported a directory as a procedure.
6071 (`#3793 <https://github.com/datalad/datalad/issues/3793>`__)
6072
6073 - The cleanup for batched git-annex processes has been improved.
6074 (`#3794 <https://github.com/datalad/datalad/issues/3794>`__)
6075 (`#3851 <https://github.com/datalad/datalad/issues/3851>`__)
6076
6077 - The function for adding a version ID to an AWS S3 URL doesn’t support
6078 URLs with an “s3://” scheme and raises a ``NotImplementedError``
6079 exception when it encounters one. The function learned to return a
6080 URL untouched if an “s3://” URL comes in with a version ID.
6081 (`#3842 <https://github.com/datalad/datalad/issues/3842>`__)
6082
6083 - A few spots needed to be adjusted for compatibility with git-annex’s
6084 new ``--sameas``
6085 `feature <https://git-annex.branchable.com/tips/multiple_remotes_accessing_the_same_data_store/>`__,
6086 which allows special remotes to share a data store.
6087 (`#3856 <https://github.com/datalad/datalad/issues/3856>`__)
6088
6089 - The ``swallow_logs`` utility failed to capture some log messages due
6090 to an incompatibility with Python 3.7.
6091 (`#3935 <https://github.com/datalad/datalad/issues/3935>`__)
6092
6093 - `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
6094
6095 - crashed if ``--inherit`` was passed but the parent dataset did not
6096 have a remote with a matching name.
6097 (`#3954 <https://github.com/datalad/datalad/issues/3954>`__)
6098 - configured the wrong pushurl and annexurl values in some cases.
6099 (`#3955 <https://github.com/datalad/datalad/issues/3955>`__)
6100
6101 Enhancements and new features since 0.12.0rc6
6102 ---------------------------------------------
6103
6104 - By default, datasets cloned from local source paths will now get a
6105 configured remote for any recursively discoverable ‘origin’ sibling
6106 that is also available from a local path in order to maximize
6107 automatic file availability across local annexes.
6108 (`#3926 <https://github.com/datalad/datalad/issues/3926>`__)
6109
6110 - The new `result hooks
6111 mechanism <http://handbook.datalad.org/en/latest/basics/101-145-hooks.html>`__
6112 allows callers to specify, via local Git configuration values,
6113 DataLad command calls that will be triggered in response to matching
6114 result records (i.e., what you see when you call a command with
6115 ``-f json_pp``).
6116 (`#3903 <https://github.com/datalad/datalad/issues/3903>`__)
6117
6118 - The command interface classes learned to use a new ``_examples_``
6119 attribute to render documentation examples for both the Python and
6120 command-line API.
6121 (`#3821 <https://github.com/datalad/datalad/issues/3821>`__)
6122
6123 - Candidate URLs for cloning a submodule can now be generated based on
6124 configured templates that have access to various properties of the
6125 submodule, including its dataset ID.
6126 (`#3828 <https://github.com/datalad/datalad/issues/3828>`__)
6127
6128 - DataLad’s check that the user’s Git identity is configured has been
6129 sped up and now considers the appropriate environment variables as
6130 well. (`#3807 <https://github.com/datalad/datalad/issues/3807>`__)
6131
6132 - The ``tag`` method of ``GitRepo`` can now tag revisions other than
6133 ``HEAD`` and accepts a list of arbitrary ``git tag`` options.
6134 (`#3787 <https://github.com/datalad/datalad/issues/3787>`__)
6135
6136 - When ``get`` clones a subdataset and the subdataset’s HEAD differs
6137 from the commit that is registered in the parent, the active branch
6138 of the subdataset is moved to the registered commit if the registered
6139 commit is an ancestor of the subdataset’s HEAD commit. This handling
6140 has been moved to a more central location within ``GitRepo``, and now
6141 applies to any ``update_submodule(..., init=True)`` call.
6142 (`#3831 <https://github.com/datalad/datalad/issues/3831>`__)
6143
6144 - The output of ``datalad -h`` has been reformatted to improve
6145 readability.
6146 (`#3862 <https://github.com/datalad/datalad/issues/3862>`__)
6147
6148 - `unlock <http://datalad.readthedocs.io/en/latest/generated/man/datalad-unlock.html>`__
6149 has been sped up.
6150 (`#3880 <https://github.com/datalad/datalad/issues/3880>`__)
6151
6152 - `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
6153 learned to provide and render more information about discovered
6154 procedures, including whether the procedure is overridden by another
6155 procedure with the same base name.
6156 (`#3960 <https://github.com/datalad/datalad/issues/3960>`__)
6157
6158 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
6159 now (`#3817 <https://github.com/datalad/datalad/issues/3817>`__)
6160
6161 - records the active branch in the superdataset when registering a
6162 new subdataset.
6163 - calls ``git annex sync`` when saving a dataset on an adjusted
6164 branch so that the changes are brought into the mainline branch.
6165
6166 - `subdatasets <http://datalad.readthedocs.io/en/latest/generated/man/datalad-subdatasets.html>`__
6167 now aborts when its ``dataset`` argument points to a non-existent
6168 dataset. (`#3940 <https://github.com/datalad/datalad/issues/3940>`__)
6169
6170 - `wtf <http://datalad.readthedocs.io/en/latest/generated/man/datalad-wtf.html>`__
6171 now
6172
6173 - reports the dataset ID if the current working directory is
6174 visiting a dataset.
6175 (`#3888 <https://github.com/datalad/datalad/issues/3888>`__)
6176 - outputs entries deterministically.
6177 (`#3927 <https://github.com/datalad/datalad/issues/3927>`__)
6178
6179 - The ``ConfigManager`` class
6180
6181 - learned to exclude ``.datalad/config`` as a source of
6182 configuration values, restricting the sources to standard Git
6183 configuration files, when called with ``source="local"``.
6184 (`#3907 <https://github.com/datalad/datalad/issues/3907>`__)
6185 - accepts a value of “override” for its ``where`` argument to allow
6186 Python callers to more convenient override configuration.
6187 (`#3970 <https://github.com/datalad/datalad/issues/3970>`__)
6188
6189 - Commands now accept a ``dataset`` value of “^.” as shorthand for “the
6190 dataset to which the current directory belongs”.
6191 (`#3242 <https://github.com/datalad/datalad/issues/3242>`__)
6192
6193 0.12.0rc6 (Oct 19, 2019) – some releases are better than the others
6194 ===================================================================
6195
6196 bet we will fix some bugs and make a world even a better place.
6197
6198 .. _major-refactoring-and-deprecations-5:
6199
6200 Major refactoring and deprecations
6201 ----------------------------------
6202
6203 - DataLad no longer supports Python 2. The minimum supported version of
6204 Python is now 3.5.
6205 (`#3629 <https://github.com/datalad/datalad/issues/3629>`__)
6206
6207 - Much of the user-focused content at http://docs.datalad.org has been
6208 removed in favor of more up to date and complete material available
6209 in the `DataLad Handbook <http://handbook.datalad.org>`__. Going
6210 forward, the plan is to restrict http://docs.datalad.org to technical
6211 documentation geared at developers.
6212 (`#3678 <https://github.com/datalad/datalad/issues/3678>`__)
6213
6214 - `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
6215 used to allow the caller to specify which dataset(s) to update as a
6216 ``PATH`` argument or via the the ``--dataset`` option; now only the
6217 latter is supported. Path arguments only serve to restrict which
6218 subdataset are updated when operating recursively.
6219 (`#3700 <https://github.com/datalad/datalad/issues/3700>`__)
6220
6221 - Result records from a
6222 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
6223 call no longer have a “state” key.
6224 (`#3746 <https://github.com/datalad/datalad/issues/3746>`__)
6225
6226 - `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
6227 and
6228 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
6229 no longer support operating on independent hierarchies of datasets.
6230 (`#3700 <https://github.com/datalad/datalad/issues/3700>`__)
6231 (`#3746 <https://github.com/datalad/datalad/issues/3746>`__)
6232
6233 - The
6234 `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
6235 update in 0.12.0rc4 for the new path resolution logic broke the
6236 handling of inputs and outputs for calls from a subdirectory.
6237 (`#3747 <https://github.com/datalad/datalad/issues/3747>`__)
6238
6239 - The ``is_submodule_modified`` method of ``GitRepo`` as well as two
6240 helper functions in gitrepo.py, ``kwargs_to_options`` and
6241 ``split_remote_branch``, were no longer used internally or in any
6242 known DataLad extensions and have been removed.
6243 (`#3702 <https://github.com/datalad/datalad/issues/3702>`__)
6244 (`#3704 <https://github.com/datalad/datalad/issues/3704>`__)
6245
6246 - The ``only_remote`` option of ``GitRepo.is_with_annex`` was not used
6247 internally or in any known extensions and has been dropped.
6248 (`#3768 <https://github.com/datalad/datalad/issues/3768>`__)
6249
6250 - The ``get_tags`` method of ``GitRepo`` used to sort tags by committer
6251 date. It now sorts them by the tagger date for annotated tags and the
6252 committer date for lightweight tags.
6253 (`#3715 <https://github.com/datalad/datalad/issues/3715>`__)
6254
6255 - The ``rev_resolve_path`` substituted ``resolve_path`` helper.
6256 (`#3797 <https://github.com/datalad/datalad/issues/3797>`__)
6257
6258 .. _fixes-21:
6259
6260 Fixes
6261 -----
6262
6263 - Correctly handle relative paths in
6264 `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__.
6265 (`#3799 <https://github.com/datalad/datalad/issues/3799>`__)
6266 (`#3102 <https://github.com/datalad/datalad/issues/3102>`__)
6267
6268 - Do not erroneously discover directory as a procedure.
6269 (`#3793 <https://github.com/datalad/datalad/issues/3793>`__)
6270
6271 - Correctly extract version from manpage to trigger use of manpages for
6272 ``--help``.
6273 (`#3798 <https://github.com/datalad/datalad/issues/3798>`__)
6274
6275 - The ``cfg_yoda`` procedure saved all modifications in the repository
6276 rather than saving only the files it modified.
6277 (`#3680 <https://github.com/datalad/datalad/issues/3680>`__)
6278
6279 - Some spots in the documentation that were supposed appear as two
6280 hyphens were incorrectly rendered in the HTML output en-dashs.
6281 (`#3692 <https://github.com/datalad/datalad/issues/3692>`__)
6282
6283 - `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__,
6284 `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__,
6285 and
6286 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
6287 treated paths as relative to the dataset even when the string form
6288 was given, violating the new path handling rules.
6289 (`#3749 <https://github.com/datalad/datalad/issues/3749>`__)
6290 (`#3777 <https://github.com/datalad/datalad/issues/3777>`__)
6291 (`#3780 <https://github.com/datalad/datalad/issues/3780>`__)
6292
6293 - Providing the “^” shortcut to ``--dataset`` didn’t work properly when
6294 called from a subdirectory of a subdataset.
6295 (`#3772 <https://github.com/datalad/datalad/issues/3772>`__)
6296
6297 - We failed to propagate some errors from git-annex when working with
6298 its JSON output.
6299 (`#3751 <https://github.com/datalad/datalad/issues/3751>`__)
6300
6301 - With the Python API, callers are allowed to pass a string or list of
6302 strings as the ``cfg_proc`` argument to
6303 `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__,
6304 but the string form was mishandled.
6305 (`#3761 <https://github.com/datalad/datalad/issues/3761>`__)
6306
6307 - Incorrect command quoting for SSH calls on Windows that rendered
6308 basic SSH-related functionality (e.g.,
6309 `sshrun <http://datalad.readthedocs.io/en/latest/generated/man/datalad-sshrun.html>`__)
6310 on Windows unusable.
6311 (`#3688 <https://github.com/datalad/datalad/issues/3688>`__)
6312
6313 - Annex JSON result handling assumed platform-specific paths on Windows
6314 instead of the POSIX-style that is happening across all platforms.
6315 (`#3719 <https://github.com/datalad/datalad/issues/3719>`__)
6316
6317 - ``path_is_under()`` was incapable of comparing Windows paths with
6318 different drive letters.
6319 (`#3728 <https://github.com/datalad/datalad/issues/3728>`__)
6320
6321 .. _enhancements-and-new-features-24:
6322
6323 Enhancements and new features
6324 -----------------------------
6325
6326 - Provide a collection of “public” ``call_git*`` helpers within GitRepo
6327 and replace use of “private” and less specific
6328 ``_git_custom_command`` calls.
6329 (`#3791 <https://github.com/datalad/datalad/issues/3791>`__)
6330
6331 - `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__
6332 gained a ``--report-filetype``. Setting it to “raw” can give a
6333 performance boost for the price of no longer distinguishing symlinks
6334 that point to annexed content from other symlinks.
6335 (`#3701 <https://github.com/datalad/datalad/issues/3701>`__)
6336
6337 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
6338 disables file type reporting by
6339 `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__
6340 to improve performance.
6341 (`#3712 <https://github.com/datalad/datalad/issues/3712>`__)
6342
6343 - `subdatasets <http://datalad.readthedocs.io/en/latest/generated/man/datalad-subdatasets.html>`__
6344 (`#3743 <https://github.com/datalad/datalad/issues/3743>`__)
6345
6346 - now extends its result records with a ``contains`` field that
6347 lists which ``contains`` arguments matched a given subdataset.
6348 - yields an ‘impossible’ result record when a ``contains`` argument
6349 wasn’t matched to any of the reported subdatasets.
6350
6351 - `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
6352 now shows more readable output when cloning fails.
6353 (`#3775 <https://github.com/datalad/datalad/issues/3775>`__)
6354
6355 - ``SSHConnection`` now displays a more informative error message when
6356 it cannot start the ``ControlMaster`` process.
6357 (`#3776 <https://github.com/datalad/datalad/issues/3776>`__)
6358
6359 - If the new configuration option ``datalad.log.result-level`` is set
6360 to a single level, all result records will be logged at that level.
6361 If you’ve been bothered by DataLad’s double reporting of failures,
6362 consider setting this to “debug”.
6363 (`#3754 <https://github.com/datalad/datalad/issues/3754>`__)
6364
6365 - Configuration values from ``datalad -c OPTION=VALUE ...`` are now
6366 validated to provide better errors.
6367 (`#3695 <https://github.com/datalad/datalad/issues/3695>`__)
6368
6369 - `rerun <https://datalad.readthedocs.io/en/latest/generated/man/datalad-rerun.html>`__
6370 learned how to handle history with merges. As was already the case
6371 when cherry picking non-run commits, re-creating merges may results
6372 in conflicts, and ``rerun`` does not yet provide an interface to let
6373 the user handle these.
6374 (`#2754 <https://github.com/datalad/datalad/issues/2754>`__)
6375
6376 - The ``fsck`` method of ``AnnexRepo`` has been enhanced to expose more
6377 features of the underlying ``git fsck`` command.
6378 (`#3693 <https://github.com/datalad/datalad/issues/3693>`__)
6379
6380 - ``GitRepo`` now has a ``for_each_ref_`` method that wraps
6381 ``git for-each-ref``, which is used in various spots that used to
6382 rely on GitPython functionality.
6383 (`#3705 <https://github.com/datalad/datalad/issues/3705>`__)
6384
6385 - Do not pretend to be able to work in optimized (``python -O``) mode,
6386 crash early with an informative message.
6387 (`#3803 <https://github.com/datalad/datalad/issues/3803>`__)
6388
6389 0.12.0rc5 (September 04, 2019) – .
6390 ==================================
6391
6392 Various fixes and enhancements that bring the 0.12.0 release closer.
6393
6394 .. _major-refactoring-and-deprecations-6:
6395
6396 Major refactoring and deprecations
6397 ----------------------------------
6398
6399 - The two modules below have a new home. The old locations still exist
6400 as compatibility shims and will be removed in a future release.
6401
6402 - ``datalad.distribution.subdatasets`` has been moved to
6403 ``datalad.local.subdatasets``
6404 (`#3429 <https://github.com/datalad/datalad/issues/3429>`__)
6405 - ``datalad.interface.run`` has been moved to
6406 ``datalad.core.local.run``
6407 (`#3444 <https://github.com/datalad/datalad/issues/3444>`__)
6408
6409 - The ``lock`` method of ``AnnexRepo`` and the ``options`` parameter of
6410 ``AnnexRepo.unlock`` were unused internally and have been removed.
6411 (`#3459 <https://github.com/datalad/datalad/issues/3459>`__)
6412
6413 - The ``get_submodules`` method of ``GitRepo`` has been rewritten
6414 without GitPython. When the new ``compat`` flag is true (the current
6415 default), the method returns a value that is compatible with the old
6416 return value. This backwards-compatible return value and the
6417 ``compat`` flag will be removed in a future release.
6418 (`#3508 <https://github.com/datalad/datalad/issues/3508>`__)
6419
6420 - The logic for resolving relative paths given to a command has changed
6421 (`#3435 <https://github.com/datalad/datalad/issues/3435>`__). The new
6422 rule is that relative paths are taken as relative to the dataset only
6423 if a dataset *instance* is passed by the caller. In all other
6424 scenarios they’re considered relative to the current directory.
6425
6426 The main user-visible difference from the command line is that using
6427 the ``--dataset`` argument does *not* result in relative paths being
6428 taken as relative to the specified dataset. (The undocumented
6429 distinction between “rel/path” and “./rel/path” no longer exists.)
6430
6431 All commands under ``datalad.core`` and ``datalad.local``, as well as
6432 ``unlock`` and ``addurls``, follow the new logic. The goal is for all
6433 commands to eventually do so.
6434
6435 .. _fixes-22:
6436
6437 Fixes
6438 -----
6439
6440 - The function for loading JSON streams wasn’t clever enough to handle
6441 content that included a Unicode line separator like U2028.
6442 (`#3524 <https://github.com/datalad/datalad/issues/3524>`__)
6443
6444 - When
6445 `unlock <http://datalad.readthedocs.io/en/latest/generated/man/datalad-unlock.html>`__
6446 was called without an explicit target (i.e., a directory or no paths
6447 at all), the call failed if any of the files did not have content
6448 present. (`#3459 <https://github.com/datalad/datalad/issues/3459>`__)
6449
6450 - ``AnnexRepo.get_content_info`` failed in the rare case of a key
6451 without size information.
6452 (`#3534 <https://github.com/datalad/datalad/issues/3534>`__)
6453
6454 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
6455 ignored ``--on-failure`` in its underlying call to
6456 `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__.
6457 (`#3470 <https://github.com/datalad/datalad/issues/3470>`__)
6458
6459 - Calling
6460 `remove <http://datalad.readthedocs.io/en/latest/generated/man/datalad-remove.html>`__
6461 with a subdirectory displayed spurious warnings about the
6462 subdirectory files not existing.
6463 (`#3586 <https://github.com/datalad/datalad/issues/3586>`__)
6464
6465 - Our processing of ``git-annex --json`` output mishandled info
6466 messages from special remotes.
6467 (`#3546 <https://github.com/datalad/datalad/issues/3546>`__)
6468
6469 - `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__
6470
6471 - didn’t bypass the “existing subdataset” check when called with
6472 ``--force`` as of 0.12.0rc3
6473 (`#3552 <https://github.com/datalad/datalad/issues/3552>`__)
6474 - failed to register the up-to-date revision of a subdataset when
6475 ``--cfg-proc`` was used with ``--dataset``
6476 (`#3591 <https://github.com/datalad/datalad/issues/3591>`__)
6477
6478 - The base downloader had some error handling that wasn’t compatible
6479 with Python 3.
6480 (`#3622 <https://github.com/datalad/datalad/issues/3622>`__)
6481
6482 - Fixed a number of Unicode py2-compatibility issues.
6483 (`#3602 <https://github.com/datalad/datalad/issues/3602>`__)
6484
6485 - ``AnnexRepo.get_content_annexinfo`` did not properly chunk file
6486 arguments to avoid exceeding the command-line character limit.
6487 (`#3587 <https://github.com/datalad/datalad/issues/3587>`__)
6488
6489 .. _enhancements-and-new-features-25:
6490
6491 Enhancements and new features
6492 -----------------------------
6493
6494 - New command ``create-sibling-gitlab`` provides an interface for
6495 creating a publication target on a GitLab instance.
6496 (`#3447 <https://github.com/datalad/datalad/issues/3447>`__)
6497
6498 - `subdatasets <http://datalad.readthedocs.io/en/latest/generated/man/datalad-subdatasets.html>`__
6499 (`#3429 <https://github.com/datalad/datalad/issues/3429>`__)
6500
6501 - now supports path-constrained queries in the same manner as
6502 commands like ``save`` and ``status``
6503 - gained a ``--contains=PATH`` option that can be used to restrict
6504 the output to datasets that include a specific path.
6505 - now narrows the listed subdatasets to those underneath the current
6506 directory when called with no arguments
6507
6508 - `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__
6509 learned to accept a plain ``--annex`` (no value) as shorthand for
6510 ``--annex basic``.
6511 (`#3534 <https://github.com/datalad/datalad/issues/3534>`__)
6512
6513 - The ``.dirty`` property of ``GitRepo`` and ``AnnexRepo`` has been
6514 sped up. (`#3460 <https://github.com/datalad/datalad/issues/3460>`__)
6515
6516 - The ``get_content_info`` method of ``GitRepo``, used by ``status``
6517 and commands that depend on ``status``, now restricts its git calls
6518 to a subset of files, if possible, for a performance gain in
6519 repositories with many files.
6520 (`#3508 <https://github.com/datalad/datalad/issues/3508>`__)
6521
6522 - Extensions that do not provide a command, such as those that provide
6523 only metadata extractors, are now supported.
6524 (`#3531 <https://github.com/datalad/datalad/issues/3531>`__)
6525
6526 - When calling git-annex with ``--json``, we log standard error at the
6527 debug level rather than the warning level if a non-zero exit is
6528 expected behavior.
6529 (`#3518 <https://github.com/datalad/datalad/issues/3518>`__)
6530
6531 - `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__
6532 no longer refuses to create a new dataset in the odd scenario of an
6533 empty .git/ directory upstairs.
6534 (`#3475 <https://github.com/datalad/datalad/issues/3475>`__)
6535
6536 - As of v2.22.0 Git treats a sub-repository on an unborn branch as a
6537 repository rather than as a directory. Our documentation and tests
6538 have been updated appropriately.
6539 (`#3476 <https://github.com/datalad/datalad/issues/3476>`__)
6540
6541 - `addurls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-addurls.html>`__
6542 learned to accept a ``--cfg-proc`` value and pass it to its
6543 ``create`` calls.
6544 (`#3562 <https://github.com/datalad/datalad/issues/3562>`__)
6545
6546 0.12.0rc4 (May 15, 2019) – the revolution is over
6547 =================================================
6548
6549 With the replacement of the ``save`` command implementation with
6550 ``rev-save`` the revolution effort is now over, and the set of key
6551 commands for local dataset operations (``create``, ``run``, ``save``,
6552 ``status``, ``diff``) is now complete. This new core API is available
6553 from ``datalad.core.local`` (and also via ``datalad.api``, as any other
6554 command).
6555
6556 .. _major-refactoring-and-deprecations-7:
6557
6558 Major refactoring and deprecations
6559 ----------------------------------
6560
6561 - The ``add`` command is now deprecated. It will be removed in a future
6562 release.
6563
6564 .. _fixes-23:
6565
6566 Fixes
6567 -----
6568
6569 - Remove hard-coded dependencies on POSIX path conventions in SSH
6570 support code
6571 (`#3400 <https://github.com/datalad/datalad/issues/3400>`__)
6572
6573 - Emit an ``add`` result when adding a new subdataset during
6574 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
6575 (`#3398 <https://github.com/datalad/datalad/issues/3398>`__)
6576
6577 - SSH file transfer now actually opens a shared connection, if none
6578 exists yet
6579 (`#3403 <https://github.com/datalad/datalad/issues/3403>`__)
6580
6581 .. _enhancements-and-new-features-26:
6582
6583 Enhancements and new features
6584 -----------------------------
6585
6586 - ``SSHConnection`` now offers methods for file upload and download
6587 (``get()``, ``put()``. The previous ``copy()`` method only supported
6588 upload and was discontinued
6589 (`#3401 <https://github.com/datalad/datalad/issues/3401>`__)
6590
6591 0.12.0rc3 (May 07, 2019) – the revolution continues
6592 ===================================================
6593
6594 Continues API consolidation and replaces the ``create`` and ``diff``
6595 command with more performant implementations.
6596
6597 .. _major-refactoring-and-deprecations-8:
6598
6599 Major refactoring and deprecations
6600 ----------------------------------
6601
6602 - The previous ``diff`` command has been replaced by the diff variant
6603 from the
6604 `datalad-revolution <http://github.com/datalad/datalad-revolution>`__
6605 extension.
6606 (`#3366 <https://github.com/datalad/datalad/issues/3366>`__)
6607
6608 - ``rev-create`` has been renamed to ``create``, and the previous
6609 ``create`` has been removed.
6610 (`#3383 <https://github.com/datalad/datalad/issues/3383>`__)
6611
6612 - The procedure ``setup_yoda_dataset`` has been renamed to ``cfg_yoda``
6613 (`#3353 <https://github.com/datalad/datalad/issues/3353>`__).
6614
6615 - The ``--nosave`` of ``addurls`` now affects only added content, not
6616 newly created subdatasets
6617 (`#3259 <https://github.com/datalad/datalad/issues/3259>`__).
6618
6619 - ``Dataset.get_subdatasets`` (deprecated since v0.9.0) has been
6620 removed. (`#3336 <https://github.com/datalad/datalad/issues/3336>`__)
6621
6622 - The ``.is_dirty`` method of ``GitRepo`` and ``AnnexRepo`` has been
6623 replaced by ``.status`` or, for a subset of cases, the ``.dirty``
6624 property.
6625 (`#3330 <https://github.com/datalad/datalad/issues/3330>`__)
6626
6627 - ``AnnexRepo.get_status`` has been replaced by ``AnnexRepo.status``.
6628 (`#3330 <https://github.com/datalad/datalad/issues/3330>`__)
6629
6630 .. _fixes-24:
6631
6632 Fixes
6633 -----
6634
6635 - `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__
6636
6637 - reported on directories that contained only ignored files
6638 (`#3238 <https://github.com/datalad/datalad/issues/3238>`__)
6639 - gave a confusing failure when called from a subdataset with an
6640 explicitly specified dataset argument and “.” as a path
6641 (`#3325 <https://github.com/datalad/datalad/issues/3325>`__)
6642 - misleadingly claimed that the locally present content size was
6643 zero when ``--annex basic`` was specified
6644 (`#3378 <https://github.com/datalad/datalad/issues/3378>`__)
6645
6646 - An informative error wasn’t given when a download provider was
6647 invalid. (`#3258 <https://github.com/datalad/datalad/issues/3258>`__)
6648
6649 - Calling ``rev-save PATH`` saved unspecified untracked subdatasets.
6650 (`#3288 <https://github.com/datalad/datalad/issues/3288>`__)
6651
6652 - The available choices for command-line options that take values are
6653 now displayed more consistently in the help output.
6654 (`#3326 <https://github.com/datalad/datalad/issues/3326>`__)
6655
6656 - The new pathlib-based code had various encoding issues on Python 2.
6657 (`#3332 <https://github.com/datalad/datalad/issues/3332>`__)
6658
6659 .. _enhancements-and-new-features-27:
6660
6661 Enhancements and new features
6662 -----------------------------
6663
6664 - `wtf <http://datalad.readthedocs.io/en/latest/generated/man/datalad-wtf.html>`__
6665 now includes information about the Python version.
6666 (`#3255 <https://github.com/datalad/datalad/issues/3255>`__)
6667
6668 - When operating in an annex repository, checking whether git-annex is
6669 available is now delayed until a call to git-annex is actually
6670 needed, allowing systems without git-annex to operate on annex
6671 repositories in a restricted fashion.
6672 (`#3274 <https://github.com/datalad/datalad/issues/3274>`__)
6673
6674 - The ``load_stream`` on helper now supports auto-detection of
6675 compressed files.
6676 (`#3289 <https://github.com/datalad/datalad/issues/3289>`__)
6677
6678 - ``create`` (formerly ``rev-create``)
6679
6680 - learned to be speedier by passing a path to ``status``
6681 (`#3294 <https://github.com/datalad/datalad/issues/3294>`__)
6682 - gained a ``--cfg-proc`` (or ``-c``) convenience option for running
6683 configuration procedures (or more accurately any procedure that
6684 begins with “cfg\_”) in the newly created dataset
6685 (`#3353 <https://github.com/datalad/datalad/issues/3353>`__)
6686
6687 - ``AnnexRepo.set_metadata`` now returns a list while
6688 ``AnnexRepo.set_metadata_`` returns a generator, a behavior which is
6689 consistent with the ``add`` and ``add_`` method pair.
6690 (`#3298 <https://github.com/datalad/datalad/issues/3298>`__)
6691
6692 - ``AnnexRepo.get_metadata`` now supports batch querying of known annex
6693 files. Note, however, that callers should carefully validate the
6694 input paths because the batch call will silently hang if given
6695 non-annex files.
6696 (`#3364 <https://github.com/datalad/datalad/issues/3364>`__)
6697
6698 - `status <http://datalad.readthedocs.io/en/latest/generated/man/datalad-status.html>`__
6699
6700 - now reports a “bytesize” field for files tracked by Git
6701 (`#3299 <https://github.com/datalad/datalad/issues/3299>`__)
6702 - gained a new option ``eval_subdataset_state`` that controls how
6703 the subdataset state is evaluated. Depending on the information
6704 you need, you can select a less expensive mode to make ``status``
6705 faster.
6706 (`#3324 <https://github.com/datalad/datalad/issues/3324>`__)
6707 - colors deleted files “red”
6708 (`#3334 <https://github.com/datalad/datalad/issues/3334>`__)
6709
6710 - Querying repository content is faster due to batching of
6711 ``git cat-file`` calls.
6712 (`#3301 <https://github.com/datalad/datalad/issues/3301>`__)
6713
6714 - The dataset ID of a subdataset is now recorded in the superdataset.
6715 (`#3304 <https://github.com/datalad/datalad/issues/3304>`__)
6716
6717 - ``GitRepo.diffstatus``
6718
6719 - now avoids subdataset recursion when the comparison is not with
6720 the working tree, which substantially improves performance when
6721 diffing large dataset hierarchies
6722 (`#3314 <https://github.com/datalad/datalad/issues/3314>`__)
6723 - got smarter and faster about labeling a subdataset as “modified”
6724 (`#3343 <https://github.com/datalad/datalad/issues/3343>`__)
6725
6726 - ``GitRepo.get_content_info`` now supports disabling the file type
6727 evaluation, which gives a performance boost in cases where this
6728 information isn’t needed.
6729 (`#3362 <https://github.com/datalad/datalad/issues/3362>`__)
6730
6731 - The XMP metadata extractor now filters based on file name to improve
6732 its performance.
6733 (`#3329 <https://github.com/datalad/datalad/issues/3329>`__)
6734
6735 0.12.0rc2 (Mar 18, 2019) – revolution!
6736 ======================================
6737
6738 .. _fixes-25:
6739
6740 Fixes
6741 -----
6742
6743 - ``GitRepo.dirty`` does not report on nested empty directories
6744 (`#3196 <https://github.com/datalad/datalad/issues/3196>`__).
6745
6746 - ``GitRepo.save()`` reports results on deleted files.
6747
6748 .. _enhancements-and-new-features-28:
6749
6750 Enhancements and new features
6751 -----------------------------
6752
6753 - Absorb a new set of core commands from the datalad-revolution
6754 extension:
6755
6756 - ``rev-status``: like ``git status``, but simpler and working with
6757 dataset hierarchies
6758 - ``rev-save``: a 2-in-1 replacement for save and add
6759 - ``rev-create``: a ~30% faster create
6760
6761 - JSON support tools can now read and write compressed files.
6762
6763 0.12.0rc1 (Mar 03, 2019) – to boldly go …
6764 =========================================
6765
6766 .. _major-refactoring-and-deprecations-9:
6767
6768 Major refactoring and deprecations
6769 ----------------------------------
6770
6771 - Discontinued support for git-annex direct-mode (also no longer
6772 supported upstream).
6773
6774 .. _enhancements-and-new-features-29:
6775
6776 Enhancements and new features
6777 -----------------------------
6778
6779 - Dataset and Repo object instances are now hashable, and can be
6780 created based on pathlib Path object instances
6781
6782 - Imported various additional methods for the Repo classes to query
6783 information and save changes.
6784
6785 0.11.8 (Oct 11, 2019) – annex-we-are-catching-up
6786 ================================================
6787
6788 .. _fixes-26:
6789
6790 Fixes
6791 -----
6792
6793 - Our internal command runner failed to capture output in some cases.
6794 (`#3656 <https://github.com/datalad/datalad/issues/3656>`__)
6795 - Workaround in the tests around python in cPython >= 3.7.5 ‘;’ in the
6796 filename confusing mimetypes
6797 (`#3769 <https://github.com/datalad/datalad/issues/3769>`__)
6798 (`#3770 <https://github.com/datalad/datalad/issues/3770>`__)
6799
6800 .. _enhancements-and-new-features-30:
6801
6802 Enhancements and new features
6803 -----------------------------
6804
6805 - Prepared for upstream changes in git-annex, including support for the
6806 latest git-annex
6807
6808 - 7.20190912 auto-upgrades v5 repositories to v7.
6809 (`#3648 <https://github.com/datalad/datalad/issues/3648>`__)
6810 (`#3682 <https://github.com/datalad/datalad/issues/3682>`__)
6811 - 7.20191009 fixed treatment of (larger/smaller)than in
6812 .gitattributes
6813 (`#3765 <https://github.com/datalad/datalad/issues/3765>`__)
6814
6815 - The ``cfg_text2git`` procedure, as well the ``--text-no-annex``
6816 option of
6817 `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__,
6818 now configure .gitattributes so that empty files are stored in git
6819 rather than annex.
6820 (`#3667 <https://github.com/datalad/datalad/issues/3667>`__)
6821
6822 0.11.7 (Sep 06, 2019) – python2-we-still-love-you-but-…
6823 =======================================================
6824
6825 Primarily bugfixes with some optimizations and refactorings.
6826
6827 .. _fixes-27:
6828
6829 Fixes
6830 -----
6831
6832 - `addurls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-addurls.html>`__
6833
6834 - now provides better handling when the URL file isn’t in the
6835 expected format.
6836 (`#3579 <https://github.com/datalad/datalad/issues/3579>`__)
6837 - always considered a relative file for the URL file argument as
6838 relative to the current working directory, which goes against the
6839 convention used by other commands of taking relative paths as
6840 relative to the dataset argument.
6841 (`#3582 <https://github.com/datalad/datalad/issues/3582>`__)
6842
6843 - `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
6844
6845 - hard coded “python” when formatting the command for non-executable
6846 procedures ending with “.py”. ``sys.executable`` is now used.
6847 (`#3624 <https://github.com/datalad/datalad/issues/3624>`__)
6848 - failed if arguments needed more complicated quoting than simply
6849 surrounding the value with double quotes. This has been resolved
6850 for systems that support ``shlex.quote``, but note that on Windows
6851 values are left unquoted.
6852 (`#3626 <https://github.com/datalad/datalad/issues/3626>`__)
6853
6854 - `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
6855 now displays an informative error message if a local path is given to
6856 ``--url`` but ``--name`` isn’t specified.
6857 (`#3555 <https://github.com/datalad/datalad/issues/3555>`__)
6858
6859 - `sshrun <http://datalad.readthedocs.io/en/latest/generated/man/datalad-sshrun.html>`__,
6860 the command DataLad uses for ``GIT_SSH_COMMAND``, didn’t support all
6861 the parameters that Git expects it to.
6862 (`#3616 <https://github.com/datalad/datalad/issues/3616>`__)
6863
6864 - Fixed a number of Unicode py2-compatibility issues.
6865 (`#3597 <https://github.com/datalad/datalad/issues/3597>`__)
6866
6867 - `download-url <https://datalad.readthedocs.io/en/latest/generated/man/datalad-download-url.html>`__
6868 now will create leading directories of the output path if they do not
6869 exist (`#3646 <https://github.com/datalad/datalad/issues/3646>`__)
6870
6871 .. _enhancements-and-new-features-31:
6872
6873 Enhancements and new features
6874 -----------------------------
6875
6876 - The
6877 `annotate-paths <http://docs.datalad.org/en/latest/generated/man/datalad-annotate-paths.html>`__
6878 helper now caches subdatasets it has seen to avoid unnecessary calls.
6879 (`#3570 <https://github.com/datalad/datalad/issues/3570>`__)
6880
6881 - A repeated configuration query has been dropped from the handling of
6882 ``--proc-pre`` and ``--proc-post``.
6883 (`#3576 <https://github.com/datalad/datalad/issues/3576>`__)
6884
6885 - Calls to ``git annex find`` now use ``--in=.`` instead of the alias
6886 ``--in=here`` to take advantage of an optimization that git-annex (as
6887 of the current release, 7.20190730) applies only to the former.
6888 (`#3574 <https://github.com/datalad/datalad/issues/3574>`__)
6889
6890 - `addurls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-addurls.html>`__
6891 now suggests close matches when the URL or file format contains an
6892 unknown field.
6893 (`#3594 <https://github.com/datalad/datalad/issues/3594>`__)
6894
6895 - Shared logic used in the setup.py files of DataLad and its extensions
6896 has been moved to modules in the \_datalad_build_support/ directory.
6897 (`#3600 <https://github.com/datalad/datalad/issues/3600>`__)
6898
6899 - Get ready for upcoming git-annex dropping support for direct mode
6900 (`#3631 <https://github.com/datalad/datalad/issues/3631>`__)
6901
6902 0.11.6 (Jul 30, 2019) – am I the last of 0.11.x?
6903 ================================================
6904
6905 Primarily bug fixes to achieve more robust performance
6906
6907 .. _fixes-28:
6908
6909 Fixes
6910 -----
6911
6912 - Our tests needed various adjustments to keep up with upstream changes
6913 in Travis and Git.
6914 (`#3479 <https://github.com/datalad/datalad/issues/3479>`__)
6915 (`#3492 <https://github.com/datalad/datalad/issues/3492>`__)
6916 (`#3493 <https://github.com/datalad/datalad/issues/3493>`__)
6917
6918 - ``AnnexRepo.is_special_annex_remote`` was too selective in what it
6919 considered to be a special remote.
6920 (`#3499 <https://github.com/datalad/datalad/issues/3499>`__)
6921
6922 - We now provide information about unexpected output when git-annex is
6923 called with ``--json``.
6924 (`#3516 <https://github.com/datalad/datalad/issues/3516>`__)
6925
6926 - Exception logging in the ``__del__`` method of ``GitRepo`` and
6927 ``AnnexRepo`` no longer fails if the names it needs are no longer
6928 bound. (`#3527 <https://github.com/datalad/datalad/issues/3527>`__)
6929
6930 - `addurls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-addurls.html>`__
6931 botched the construction of subdataset paths that were more than two
6932 levels deep and failed to create datasets in a reliable,
6933 breadth-first order.
6934 (`#3561 <https://github.com/datalad/datalad/issues/3561>`__)
6935
6936 - Cloning a ``type=git`` special remote showed a spurious warning about
6937 the remote not being enabled.
6938 (`#3547 <https://github.com/datalad/datalad/issues/3547>`__)
6939
6940 .. _enhancements-and-new-features-32:
6941
6942 Enhancements and new features
6943 -----------------------------
6944
6945 - For calls to git and git-annex, we disable automatic garbage
6946 collection due to past issues with GitPython’s state becoming stale,
6947 but doing so results in a larger .git/objects/ directory that isn’t
6948 cleaned up until garbage collection is triggered outside of DataLad.
6949 Tests with the latest GitPython didn’t reveal any state issues, so
6950 we’ve re-enabled automatic garbage collection.
6951 (`#3458 <https://github.com/datalad/datalad/issues/3458>`__)
6952
6953 - `rerun <https://datalad.readthedocs.io/en/latest/generated/man/datalad-rerun.html>`__
6954 learned an ``--explicit`` flag, which it relays to its calls to
6955 [run][[]]. This makes it possible to call ``rerun`` in a dirty
6956 working tree
6957 (`#3498 <https://github.com/datalad/datalad/issues/3498>`__).
6958
6959 - The
6960 `metadata <http://datalad.readthedocs.io/en/latest/generated/man/datalad-metadata.html>`__
6961 command aborts earlier if a metadata extractor is unavailable.
6962 (`#3525 <https://github.com/datalad/datalad/issues/3525>`__)
6963
6964 0.11.5 (May 23, 2019) – stability is not overrated
6965 ==================================================
6966
6967 Should be faster and less buggy, with a few enhancements.
6968
6969 .. _fixes-29:
6970
6971 Fixes
6972 -----
6973
6974 - `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
6975 (`#3318 <https://github.com/datalad/datalad/issues/3318>`__)
6976
6977 - Siblings are no longer configured with a post-update hook unless a
6978 web interface is requested with ``--ui``.
6979 - ``git submodule update --init`` is no longer called from the
6980 post-update hook.
6981 - If ``--inherit`` is given for a dataset without a superdataset, a
6982 warning is now given instead of raising an error.
6983
6984 - The internal command runner failed on Python 2 when its ``env``
6985 argument had unicode values.
6986 (`#3332 <https://github.com/datalad/datalad/issues/3332>`__)
6987 - The safeguard that prevents creating a dataset in a subdirectory that
6988 already contains tracked files for another repository failed on Git
6989 versions before 2.14. For older Git versions, we now warn the caller
6990 that the safeguard is not active.
6991 (`#3347 <https://github.com/datalad/datalad/issues/3347>`__)
6992 - A regression introduced in v0.11.1 prevented
6993 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
6994 from committing changes under a subdirectory when the subdirectory
6995 was specified as a path argument.
6996 (`#3106 <https://github.com/datalad/datalad/issues/3106>`__)
6997 - A workaround introduced in v0.11.1 made it possible for
6998 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
6999 to do a partial commit with an annex file that has gone below the
7000 ``annex.largefiles`` threshold. The logic of this workaround was
7001 faulty, leading to files being displayed as typechanged in the index
7002 following the commit.
7003 (`#3365 <https://github.com/datalad/datalad/issues/3365>`__)
7004 - The resolve_path() helper confused paths that had a semicolon for SSH
7005 RIs. (`#3425 <https://github.com/datalad/datalad/issues/3425>`__)
7006 - The detection of SSH RIs has been improved.
7007 (`#3425 <https://github.com/datalad/datalad/issues/3425>`__)
7008
7009 .. _enhancements-and-new-features-33:
7010
7011 Enhancements and new features
7012 -----------------------------
7013
7014 - The internal command runner was too aggressive in its decision to
7015 sleep. (`#3322 <https://github.com/datalad/datalad/issues/3322>`__)
7016 - The “INFO” label in log messages now retains the default text color
7017 for the terminal rather than using white, which only worked well for
7018 terminals with dark backgrounds.
7019 (`#3334 <https://github.com/datalad/datalad/issues/3334>`__)
7020 - A short flag ``-R`` is now available for the ``--recursion-limit``
7021 flag, a flag shared by several subcommands.
7022 (`#3340 <https://github.com/datalad/datalad/issues/3340>`__)
7023 - The authentication logic for
7024 `create-sibling-github <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-github.html>`__
7025 has been revamped and now supports 2FA.
7026 (`#3180 <https://github.com/datalad/datalad/issues/3180>`__)
7027 - New configuration option ``datalad.ui.progressbar`` can be used to
7028 configure the default backend for progress reporting (“none”, for
7029 example, results in no progress bars being shown).
7030 (`#3396 <https://github.com/datalad/datalad/issues/3396>`__)
7031 - A new progress backend, available by setting datalad.ui.progressbar
7032 to “log”, replaces progress bars with a log message upon completion
7033 of an action.
7034 (`#3396 <https://github.com/datalad/datalad/issues/3396>`__)
7035 - DataLad learned to consult the `NO_COLOR <https://no-color.org/>`__
7036 environment variable and the new ``datalad.ui.color`` configuration
7037 option when deciding to color output. The default value, “auto”,
7038 retains the current behavior of coloring output if attached to a TTY
7039 (`#3407 <https://github.com/datalad/datalad/issues/3407>`__).
7040 - `clean <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clean.html>`__
7041 now removes annex transfer directories, which is useful for cleaning
7042 up failed downloads.
7043 (`#3374 <https://github.com/datalad/datalad/issues/3374>`__)
7044 - `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
7045 no longer refuses to clone into a local path that looks like a URL,
7046 making its behavior consistent with ``git clone``.
7047 (`#3425 <https://github.com/datalad/datalad/issues/3425>`__)
7048 - `wtf <http://datalad.readthedocs.io/en/latest/generated/man/datalad-wtf.html>`__
7049
7050 - Learned to fall back to the ``dist`` package if ``platform.dist``,
7051 which has been removed in the yet-to-be-release Python 3.8, does
7052 not exist.
7053 (`#3439 <https://github.com/datalad/datalad/issues/3439>`__)
7054 - Gained a ``--section`` option for limiting the output to specific
7055 sections and a ``--decor`` option, which currently knows how to
7056 format the output as GitHub’s ``<details>`` section.
7057 (`#3440 <https://github.com/datalad/datalad/issues/3440>`__)
7058
7059 0.11.4 (Mar 18, 2019) – get-ready
7060 =================================
7061
7062 Largely a bug fix release with a few enhancements
7063
7064 Important
7065 ---------
7066
7067 - 0.11.x series will be the last one with support for direct mode of
7068 `git-annex <http://git-annex.branchable.com/>`__ which is used on
7069 crippled (no symlinks and no locking) filesystems. v7 repositories
7070 should be used instead.
7071
7072 .. _fixes-30:
7073
7074 Fixes
7075 -----
7076
7077 - Extraction of .gz files is broken without p7zip installed. We now
7078 abort with an informative error in this situation.
7079 (`#3176 <https://github.com/datalad/datalad/issues/3176>`__)
7080
7081 - Committing failed in some cases because we didn’t ensure that the
7082 path passed to ``git read-tree --index-output=...`` resided on the
7083 same filesystem as the repository.
7084 (`#3181 <https://github.com/datalad/datalad/issues/3181>`__)
7085
7086 - Some pointless warnings during metadata aggregation have been
7087 eliminated.
7088 (`#3186 <https://github.com/datalad/datalad/issues/3186>`__)
7089
7090 - With Python 3 the LORIS token authenticator did not properly decode a
7091 response
7092 (`#3205 <https://github.com/datalad/datalad/issues/3205>`__).
7093
7094 - With Python 3 downloaders unnecessarily decoded the response when
7095 getting the status, leading to an encoding error.
7096 (`#3210 <https://github.com/datalad/datalad/issues/3210>`__)
7097
7098 - In some cases, our internal command Runner did not adjust the
7099 environment’s ``PWD`` to match the current working directory
7100 specified with the ``cwd`` parameter.
7101 (`#3215 <https://github.com/datalad/datalad/issues/3215>`__)
7102
7103 - The specification of the pyliblzma dependency was broken.
7104 (`#3220 <https://github.com/datalad/datalad/issues/3220>`__)
7105
7106 - `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
7107 displayed an uninformative blank log message in some cases.
7108 (`#3222 <https://github.com/datalad/datalad/issues/3222>`__)
7109
7110 - The logic for finding the location of the aggregate metadata DB
7111 anchored the search path incorrectly, leading to a spurious warning.
7112 (`#3241 <https://github.com/datalad/datalad/issues/3241>`__)
7113
7114 - Some progress bars were still displayed when stdout and stderr were
7115 not attached to a tty.
7116 (`#3281 <https://github.com/datalad/datalad/issues/3281>`__)
7117
7118 - Check for stdin/out/err to not be closed before checking for
7119 ``.isatty``.
7120 (`#3268 <https://github.com/datalad/datalad/issues/3268>`__)
7121
7122 .. _enhancements-and-new-features-34:
7123
7124 Enhancements and new features
7125 -----------------------------
7126
7127 - Creating a new repository now aborts if any of the files in the
7128 directory are tracked by a repository in a parent directory.
7129 (`#3211 <https://github.com/datalad/datalad/issues/3211>`__)
7130
7131 - `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
7132 learned to replace the ``{tmpdir}`` placeholder in commands with a
7133 temporary directory.
7134 (`#3223 <https://github.com/datalad/datalad/issues/3223>`__)
7135
7136 - `duecredit <https://github.com/duecredit/duecredit>`__ support has
7137 been added for citing DataLad itself as well as datasets that an
7138 analysis uses.
7139 (`#3184 <https://github.com/datalad/datalad/issues/3184>`__)
7140
7141 - The ``eval_results`` interface helper unintentionally modified one of
7142 its arguments.
7143 (`#3249 <https://github.com/datalad/datalad/issues/3249>`__)
7144
7145 - A few DataLad constants have been added, changed, or renamed
7146 (`#3250 <https://github.com/datalad/datalad/issues/3250>`__):
7147
7148 - ``HANDLE_META_DIR`` is now ``DATALAD_DOTDIR``. The old name should
7149 be considered deprecated.
7150 - ``METADATA_DIR`` now refers to ``DATALAD_DOTDIR/metadata`` rather
7151 than ``DATALAD_DOTDIR/meta`` (which is still available as
7152 ``OLDMETADATA_DIR``).
7153 - The new ``DATASET_METADATA_FILE`` refers to
7154 ``METADATA_DIR/dataset.json``.
7155 - The new ``DATASET_CONFIG_FILE`` refers to
7156 ``DATALAD_DOTDIR/config``.
7157 - ``METADATA_FILENAME`` has been renamed to
7158 ``OLDMETADATA_FILENAME``.
7159
7160 0.11.3 (Feb 19, 2019) – read-me-gently
7161 ======================================
7162
7163 Just a few of important fixes and minor enhancements.
7164
7165 .. _fixes-31:
7166
7167 Fixes
7168 -----
7169
7170 - The logic for setting the maximum command line length now works
7171 around Python 3.4 returning an unreasonably high value for
7172 ``SC_ARG_MAX`` on Debian systems.
7173 (`#3165 <https://github.com/datalad/datalad/issues/3165>`__)
7174
7175 - DataLad commands that are conceptually “read-only”, such as
7176 ``datalad ls -L``, can fail when the caller lacks write permissions
7177 because git-annex tries merging remote git-annex branches to update
7178 information about availability. DataLad now disables
7179 ``annex.merge-annex-branches`` in some common “read-only” scenarios
7180 to avoid these failures.
7181 (`#3164 <https://github.com/datalad/datalad/issues/3164>`__)
7182
7183 .. _enhancements-and-new-features-35:
7184
7185 Enhancements and new features
7186 -----------------------------
7187
7188 - Accessing an “unbound” dataset method now automatically imports the
7189 necessary module rather than requiring an explicit import from the
7190 Python caller. For example, calling ``Dataset.add`` no longer needs
7191 to be preceded by ``from datalad.distribution.add import Add`` or an
7192 import of ``datalad.api``.
7193 (`#3156 <https://github.com/datalad/datalad/issues/3156>`__)
7194
7195 - Configuring the new variable ``datalad.ssh.identityfile`` instructs
7196 DataLad to pass a value to the ``-i`` option of ``ssh``.
7197 (`#3149 <https://github.com/datalad/datalad/issues/3149>`__)
7198 (`#3168 <https://github.com/datalad/datalad/issues/3168>`__)
7199
7200 0.11.2 (Feb 07, 2019) – live-long-and-prosper
7201 =============================================
7202
7203 A variety of bugfixes and enhancements
7204
7205 .. _major-refactoring-and-deprecations-10:
7206
7207 Major refactoring and deprecations
7208 ----------------------------------
7209
7210 - All extracted metadata is now placed under git-annex by default.
7211 Previously files smaller than 20 kb were stored in git.
7212 (`#3109 <https://github.com/datalad/datalad/issues/3109>`__)
7213 - The function ``datalad.cmd.get_runner`` has been removed.
7214 (`#3104 <https://github.com/datalad/datalad/issues/3104>`__)
7215
7216 .. _fixes-32:
7217
7218 Fixes
7219 -----
7220
7221 - Improved handling of long commands:
7222
7223 - The code that inspected ``SC_ARG_MAX`` didn’t check that the
7224 reported value was a sensible, positive number.
7225 (`#3025 <https://github.com/datalad/datalad/issues/3025>`__)
7226 - More commands that invoke ``git`` and ``git-annex`` with file
7227 arguments learned to split up the command calls when it is likely
7228 that the command would fail due to exceeding the maximum supported
7229 length.
7230 (`#3138 <https://github.com/datalad/datalad/issues/3138>`__)
7231
7232 - The ``setup_yoda_dataset`` procedure created a malformed
7233 .gitattributes line.
7234 (`#3057 <https://github.com/datalad/datalad/issues/3057>`__)
7235 - `download-url <https://datalad.readthedocs.io/en/latest/generated/man/datalad-download-url.html>`__
7236 unnecessarily tried to infer the dataset when ``--no-save`` was
7237 given. (`#3029 <https://github.com/datalad/datalad/issues/3029>`__)
7238 - `rerun <https://datalad.readthedocs.io/en/latest/generated/man/datalad-rerun.html>`__
7239 aborted too late and with a confusing message when a ref specified
7240 via ``--onto`` didn’t exist.
7241 (`#3019 <https://github.com/datalad/datalad/issues/3019>`__)
7242 - `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__:
7243
7244 - ``run`` didn’t preserve the current directory prefix (“./”) on
7245 inputs and outputs, which is problematic if the caller relies on
7246 this representation when formatting the command.
7247 (`#3037 <https://github.com/datalad/datalad/issues/3037>`__)
7248 - Fixed a number of unicode py2-compatibility issues.
7249 (`#3035 <https://github.com/datalad/datalad/issues/3035>`__)
7250 (`#3046 <https://github.com/datalad/datalad/issues/3046>`__)
7251 - To proceed with a failed command, the user was confusingly
7252 instructed to use ``save`` instead of ``add`` even though ``run``
7253 uses ``add`` underneath.
7254 (`#3080 <https://github.com/datalad/datalad/issues/3080>`__)
7255
7256 - Fixed a case where the helper class for checking external modules
7257 incorrectly reported a module as unknown.
7258 (`#3051 <https://github.com/datalad/datalad/issues/3051>`__)
7259 - `add-archive-content <https://datalad.readthedocs.io/en/latest/generated/man/datalad-add-archive-content.html>`__
7260 mishandled the archive path when the leading path contained a
7261 symlink. (`#3058 <https://github.com/datalad/datalad/issues/3058>`__)
7262 - Following denied access, the credential code failed to consider a
7263 scenario, leading to a type error rather than an appropriate error
7264 message. (`#3091 <https://github.com/datalad/datalad/issues/3091>`__)
7265 - Some tests failed when executed from a ``git worktree`` checkout of
7266 the source repository.
7267 (`#3129 <https://github.com/datalad/datalad/issues/3129>`__)
7268 - During metadata extraction, batched annex processes weren’t properly
7269 terminated, leading to issues on Windows.
7270 (`#3137 <https://github.com/datalad/datalad/issues/3137>`__)
7271 - `add <http://datalad.readthedocs.io/en/latest/generated/man/datalad-add.html>`__
7272 incorrectly handled an “invalid repository” exception when trying to
7273 add a submodule.
7274 (`#3141 <https://github.com/datalad/datalad/issues/3141>`__)
7275 - Pass ``GIT_SSH_VARIANT=ssh`` to git processes to be able to specify
7276 alternative ports in SSH urls
7277
7278 .. _enhancements-and-new-features-36:
7279
7280 Enhancements and new features
7281 -----------------------------
7282
7283 - `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
7284 learned to suggest closely matching keys if there are no hits.
7285 (`#3089 <https://github.com/datalad/datalad/issues/3089>`__)
7286 - `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
7287
7288 - gained a ``--group`` option so that the caller can specify the
7289 file system group for the repository.
7290 (`#3098 <https://github.com/datalad/datalad/issues/3098>`__)
7291 - now understands SSH URLs that have a port in them (i.e. the
7292 “ssh://[user@]host.xz[:port]/path/to/repo.git/” syntax mentioned
7293 in ``man git-fetch``).
7294 (`#3146 <https://github.com/datalad/datalad/issues/3146>`__)
7295
7296 - Interface classes can now override the default renderer for
7297 summarizing results.
7298 (`#3061 <https://github.com/datalad/datalad/issues/3061>`__)
7299 - `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__:
7300
7301 - ``--input`` and ``--output`` can now be shortened to ``-i`` and
7302 ``-o``.
7303 (`#3066 <https://github.com/datalad/datalad/issues/3066>`__)
7304 - Placeholders such as “{inputs}” are now expanded in the command
7305 that is shown in the commit message subject.
7306 (`#3065 <https://github.com/datalad/datalad/issues/3065>`__)
7307 - ``interface.run.run_command`` gained an ``extra_inputs`` argument
7308 so that wrappers like
7309 `datalad-container <https://github.com/datalad/datalad-container>`__
7310 can specify additional inputs that aren’t considered when
7311 formatting the command string.
7312 (`#3038 <https://github.com/datalad/datalad/issues/3038>`__)
7313 - “–” can now be used to separate options for ``run`` and those for
7314 the command in ambiguous cases.
7315 (`#3119 <https://github.com/datalad/datalad/issues/3119>`__)
7316
7317 - The utilities ``create_tree`` and ``ok_file_has_content`` now support
7318 “.gz” files.
7319 (`#3049 <https://github.com/datalad/datalad/issues/3049>`__)
7320 - The Singularity container for 0.11.1 now uses
7321 `nd_freeze <https://github.com/neurodebian/neurodebian/blob/master/tools/nd_freeze>`__
7322 to make its builds reproducible.
7323 - A
7324 `publications <https://datalad.readthedocs.io/en/latest/publications.html>`__
7325 page has been added to the documentation.
7326 (`#3099 <https://github.com/datalad/datalad/issues/3099>`__)
7327 - ``GitRepo.set_gitattributes`` now accepts a ``mode`` argument that
7328 controls whether the .gitattributes file is appended to (default) or
7329 overwritten.
7330 (`#3115 <https://github.com/datalad/datalad/issues/3115>`__)
7331 - ``datalad --help`` now avoids using ``man`` so that the list of
7332 subcommands is shown.
7333 (`#3124 <https://github.com/datalad/datalad/issues/3124>`__)
7334
7335 0.11.1 (Nov 26, 2018) – v7-better-than-v6
7336 =========================================
7337
7338 Rushed out bugfix release to stay fully compatible with recent
7339 `git-annex <http://git-annex.branchable.com/>`__ which introduced v7 to
7340 replace v6.
7341
7342 .. _fixes-33:
7343
7344 Fixes
7345 -----
7346
7347 - `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__:
7348 be able to install recursively into a dataset
7349 (`#2982 <https://github.com/datalad/datalad/issues/2982>`__)
7350 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__:
7351 be able to commit/save changes whenever files potentially could have
7352 swapped their storage between git and annex
7353 (`#1651 <https://github.com/datalad/datalad/issues/1651>`__)
7354 (`#2752 <https://github.com/datalad/datalad/issues/2752>`__)
7355 (`#3009 <https://github.com/datalad/datalad/issues/3009>`__)
7356 - [aggregate-metadata][]:
7357
7358 - dataset’s itself is now not “aggregated” if specific paths are
7359 provided for aggregation
7360 (`#3002 <https://github.com/datalad/datalad/issues/3002>`__). That
7361 resolves the issue of ``-r`` invocation aggregating all
7362 subdatasets of the specified dataset as well
7363 - also compare/verify the actual content checksum of aggregated
7364 metadata while considering subdataset metadata for re-aggregation
7365 (`#3007 <https://github.com/datalad/datalad/issues/3007>`__)
7366
7367 - ``annex`` commands are now chunked assuming 50% “safety margin” on
7368 the maximal command line length. Should resolve crashes while
7369 operating of too many files at ones
7370 (`#3001 <https://github.com/datalad/datalad/issues/3001>`__)
7371 - ``run`` sidecar config processing
7372 (`#2991 <https://github.com/datalad/datalad/issues/2991>`__)
7373 - no double trailing period in docs
7374 (`#2984 <https://github.com/datalad/datalad/issues/2984>`__)
7375 - correct identification of the repository with symlinks in the paths
7376 in the tests
7377 (`#2972 <https://github.com/datalad/datalad/issues/2972>`__)
7378 - re-evaluation of dataset properties in case of dataset changes
7379 (`#2946 <https://github.com/datalad/datalad/issues/2946>`__)
7380 - [text2git][] procedure to use ``ds.repo.set_gitattributes``
7381 (`#2974 <https://github.com/datalad/datalad/issues/2974>`__)
7382 (`#2954 <https://github.com/datalad/datalad/issues/2954>`__)
7383 - Switch to use plain ``os.getcwd()`` if inconsistency with env var
7384 ``$PWD`` is detected
7385 (`#2914 <https://github.com/datalad/datalad/issues/2914>`__)
7386 - Make sure that credential defined in env var takes precedence
7387 (`#2960 <https://github.com/datalad/datalad/issues/2960>`__)
7388 (`#2950 <https://github.com/datalad/datalad/issues/2950>`__)
7389
7390 .. _enhancements-and-new-features-37:
7391
7392 Enhancements and new features
7393 -----------------------------
7394
7395 - `shub://datalad/datalad:git-annex-dev <https://singularity-hub.org/containers/5663/view>`__
7396 provides a Debian buster Singularity image with build environment for
7397 `git-annex <http://git-annex.branchable.com/>`__.
7398 ``tools/bisect-git-annex`` provides a helper for running
7399 ``git bisect`` on git-annex using that Singularity container
7400 (`#2995 <https://github.com/datalad/datalad/issues/2995>`__)
7401 - Added ``.zenodo.json`` for better integration with Zenodo for
7402 citation
7403 - `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
7404 now provides names and help messages with a custom renderer for
7405 (`#2993 <https://github.com/datalad/datalad/issues/2993>`__)
7406 - Documentation: point to
7407 `datalad-revolution <http://github.com/datalad/datalad-revolution>`__
7408 extension (prototype of the greater DataLad future)
7409 - `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
7410
7411 - support injecting of a detached command
7412 (`#2937 <https://github.com/datalad/datalad/issues/2937>`__)
7413
7414 - ``annex`` metadata extractor now extracts ``annex.key`` metadata
7415 record. Should allow now to identify uses of specific files etc
7416 (`#2952 <https://github.com/datalad/datalad/issues/2952>`__)
7417 - Test that we can install from http://datasets.datalad.org
7418 - Proper rendering of ``CommandError`` (e.g. in case of “out of space”
7419 error) (`#2958 <https://github.com/datalad/datalad/issues/2958>`__)
7420
7421 0.11.0 (Oct 23, 2018) – Soon-to-be-perfect
7422 ==========================================
7423
7424 `git-annex <http://git-annex.branchable.com/>`__ 6.20180913 (or later)
7425 is now required - provides a number of fixes for v6 mode operations etc.
7426
7427 .. _major-refactoring-and-deprecations-11:
7428
7429 Major refactoring and deprecations
7430 ----------------------------------
7431
7432 - ``datalad.consts.LOCAL_CENTRAL_PATH`` constant was deprecated in
7433 favor of ``datalad.locations.default-dataset``
7434 `configuration <http://docs.datalad.org/en/latest/config.html>`__
7435 variable (`#2835 <https://github.com/datalad/datalad/issues/2835>`__)
7436
7437 Minor refactoring
7438 -----------------
7439
7440 - ``"notneeded"`` messages are no longer reported by default results
7441 renderer
7442 - `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
7443 no longer shows commit instructions upon command failure when
7444 ``explicit`` is true and no outputs are specified
7445 (`#2922 <https://github.com/datalad/datalad/issues/2922>`__)
7446 - ``get_git_dir`` moved into GitRepo
7447 (`#2886 <https://github.com/datalad/datalad/issues/2886>`__)
7448 - ``_gitpy_custom_call`` removed from GitRepo
7449 (`#2894 <https://github.com/datalad/datalad/issues/2894>`__)
7450 - ``GitRepo.get_merge_base`` argument is now called ``commitishes``
7451 instead of ``treeishes``
7452 (`#2903 <https://github.com/datalad/datalad/issues/2903>`__)
7453
7454 .. _fixes-34:
7455
7456 Fixes
7457 -----
7458
7459 - `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
7460 should not leave the dataset in non-clean state
7461 (`#2858 <https://github.com/datalad/datalad/issues/2858>`__) and some
7462 other enhancements
7463 (`#2859 <https://github.com/datalad/datalad/issues/2859>`__)
7464 - Fixed chunking of the long command lines to account for decorators
7465 and other arguments
7466 (`#2864 <https://github.com/datalad/datalad/issues/2864>`__)
7467 - Progress bar should not crash the process on some missing progress
7468 information
7469 (`#2891 <https://github.com/datalad/datalad/issues/2891>`__)
7470 - Default value for ``jobs`` set to be ``"auto"`` (not ``None``) to
7471 take advantage of possible parallel get if in ``-g`` mode
7472 (`#2861 <https://github.com/datalad/datalad/issues/2861>`__)
7473 - `wtf <http://datalad.readthedocs.io/en/latest/generated/man/datalad-wtf.html>`__
7474 must not crash if ``git-annex`` is not installed etc
7475 (`#2865 <https://github.com/datalad/datalad/issues/2865>`__),
7476 (`#2865 <https://github.com/datalad/datalad/issues/2865>`__),
7477 (`#2918 <https://github.com/datalad/datalad/issues/2918>`__),
7478 (`#2917 <https://github.com/datalad/datalad/issues/2917>`__)
7479 - Fixed paths (with spaces etc) handling while reporting annex error
7480 output (`#2892 <https://github.com/datalad/datalad/issues/2892>`__),
7481 (`#2893 <https://github.com/datalad/datalad/issues/2893>`__)
7482 - ``__del__`` should not access ``.repo`` but ``._repo`` to avoid
7483 attempts for reinstantiation etc
7484 (`#2901 <https://github.com/datalad/datalad/issues/2901>`__)
7485 - Fix up submodule ``.git`` right in ``GitRepo.add_submodule`` to avoid
7486 added submodules being non git-annex friendly
7487 (`#2909 <https://github.com/datalad/datalad/issues/2909>`__),
7488 (`#2904 <https://github.com/datalad/datalad/issues/2904>`__)
7489 - `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
7490 (`#2905 <https://github.com/datalad/datalad/issues/2905>`__)
7491
7492 - now will provide dataset into the procedure if called within
7493 dataset
7494 - will not crash if procedure is an executable without ``.py`` or
7495 ``.sh`` suffixes
7496
7497 - Use centralized ``.gitattributes`` handling while setting annex
7498 backend (`#2912 <https://github.com/datalad/datalad/issues/2912>`__)
7499 - ``GlobbedPaths.expand(..., full=True)`` incorrectly returned relative
7500 paths when called more than once
7501 (`#2921 <https://github.com/datalad/datalad/issues/2921>`__)
7502
7503 .. _enhancements-and-new-features-38:
7504
7505 Enhancements and new features
7506 -----------------------------
7507
7508 - Report progress on
7509 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
7510 when installing from “smart” git servers
7511 (`#2876 <https://github.com/datalad/datalad/issues/2876>`__)
7512 - Stale/unused ``sth_like_file_has_content`` was removed
7513 (`#2860 <https://github.com/datalad/datalad/issues/2860>`__)
7514 - Enhancements to
7515 `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
7516 to operate on “improved” metadata layouts
7517 (`#2878 <https://github.com/datalad/datalad/issues/2878>`__)
7518 - Output of ``git annex init`` operation is now logged
7519 (`#2881 <https://github.com/datalad/datalad/issues/2881>`__)
7520 - New
7521
7522 - ``GitRepo.cherry_pick``
7523 (`#2900 <https://github.com/datalad/datalad/issues/2900>`__)
7524 - ``GitRepo.format_commit``
7525 (`#2902 <https://github.com/datalad/datalad/issues/2902>`__)
7526
7527 - `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
7528 (`#2905 <https://github.com/datalad/datalad/issues/2905>`__)
7529
7530 - procedures can now recursively be discovered in subdatasets as
7531 well. The uppermost has highest priority
7532 - Procedures in user and system locations now take precedence over
7533 those in datasets.
7534
7535 0.10.3.1 (Sep 13, 2018) – Nothing-is-perfect
7536 ============================================
7537
7538 Emergency bugfix to address forgotten boost of version in
7539 ``datalad/version.py``.
7540
7541 0.10.3 (Sep 13, 2018) – Almost-perfect
7542 ======================================
7543
7544 This is largely a bugfix release which addressed many (but not yet all)
7545 issues of working with git-annex direct and version 6 modes, and
7546 operation on Windows in general. Among enhancements you will see the
7547 support of public S3 buckets (even with periods in their names), ability
7548 to configure new providers interactively, and improved ``egrep`` search
7549 backend.
7550
7551 Although we do not require with this release, it is recommended to make
7552 sure that you are using a recent ``git-annex`` since it also had a
7553 variety of fixes and enhancements in the past months.
7554
7555 .. _fixes-35:
7556
7557 Fixes
7558 -----
7559
7560 - Parsing of combined short options has been broken since DataLad
7561 v0.10.0. (`#2710 <https://github.com/datalad/datalad/issues/2710>`__)
7562 - The ``datalad save`` instructions shown by ``datalad run`` for a
7563 command with a non-zero exit were incorrectly formatted.
7564 (`#2692 <https://github.com/datalad/datalad/issues/2692>`__)
7565 - Decompression of zip files (e.g., through
7566 ``datalad add-archive-content``) failed on Python 3.
7567 (`#2702 <https://github.com/datalad/datalad/issues/2702>`__)
7568 - Windows:
7569
7570 - colored log output was not being processed by colorama.
7571 (`#2707 <https://github.com/datalad/datalad/issues/2707>`__)
7572 - more codepaths now try multiple times when removing a file to deal
7573 with latency and locking issues on Windows.
7574 (`#2795 <https://github.com/datalad/datalad/issues/2795>`__)
7575
7576 - Internal git fetch calls have been updated to work around a GitPython
7577 ``BadName`` issue.
7578 (`#2712 <https://github.com/datalad/datalad/issues/2712>`__),
7579 (`#2794 <https://github.com/datalad/datalad/issues/2794>`__)
7580 - The progress bar for annex file transferring was unable to handle an
7581 empty file.
7582 (`#2717 <https://github.com/datalad/datalad/issues/2717>`__)
7583 - ``datalad add-readme`` halted when no aggregated metadata was found
7584 rather than displaying a warning.
7585 (`#2731 <https://github.com/datalad/datalad/issues/2731>`__)
7586 - ``datalad rerun`` failed if ``--onto`` was specified and the history
7587 contained no run commits.
7588 (`#2761 <https://github.com/datalad/datalad/issues/2761>`__)
7589 - Processing of a command’s results failed on a result record with a
7590 missing value (e.g., absent field or subfield in metadata). Now the
7591 missing value is rendered as “N/A”.
7592 (`#2725 <https://github.com/datalad/datalad/issues/2725>`__).
7593 - A couple of documentation links in the “Delineation from related
7594 solutions” were misformatted.
7595 (`#2773 <https://github.com/datalad/datalad/issues/2773>`__)
7596 - With the latest git-annex, several known V6 failures are no longer an
7597 issue. (`#2777 <https://github.com/datalad/datalad/issues/2777>`__)
7598 - In direct mode, commit changes would often commit annexed content as
7599 regular Git files. A new approach fixes this and resolves a good
7600 number of known failures.
7601 (`#2770 <https://github.com/datalad/datalad/issues/2770>`__)
7602 - The reporting of command results failed if the current working
7603 directory was removed (e.g., after an unsuccessful ``install``).
7604 (`#2788 <https://github.com/datalad/datalad/issues/2788>`__)
7605 - When installing into an existing empty directory, ``datalad install``
7606 removed the directory after a failed clone.
7607 (`#2788 <https://github.com/datalad/datalad/issues/2788>`__)
7608 - ``datalad run`` incorrectly handled inputs and outputs for paths with
7609 spaces and other characters that require shell escaping.
7610 (`#2798 <https://github.com/datalad/datalad/issues/2798>`__)
7611 - Globbing inputs and outputs for ``datalad run`` didn’t work correctly
7612 if a subdataset wasn’t installed.
7613 (`#2796 <https://github.com/datalad/datalad/issues/2796>`__)
7614 - Minor (in)compatibility with git 2.19 - (no) trailing period in an
7615 error message now.
7616 (`#2815 <https://github.com/datalad/datalad/issues/2815>`__)
7617
7618 .. _enhancements-and-new-features-39:
7619
7620 Enhancements and new features
7621 -----------------------------
7622
7623 - Anonymous access is now supported for S3 and other downloaders.
7624 (`#2708 <https://github.com/datalad/datalad/issues/2708>`__)
7625 - A new interface is available to ease setting up new providers.
7626 (`#2708 <https://github.com/datalad/datalad/issues/2708>`__)
7627 - Metadata: changes to egrep mode search
7628 (`#2735 <https://github.com/datalad/datalad/issues/2735>`__)
7629
7630 - Queries in egrep mode are now case-sensitive when the query
7631 contains any uppercase letters and are case-insensitive otherwise.
7632 The new mode egrepcs can be used to perform a case-sensitive query
7633 with all lower-case letters.
7634 - Search can now be limited to a specific key.
7635 - Multiple queries (list of expressions) are evaluated using AND to
7636 determine whether something is a hit.
7637 - A single multi-field query (e.g., ``pa*:findme``) is a hit, when
7638 any matching field matches the query.
7639 - All matching key/value combinations across all (multi-field)
7640 queries are reported in the query_matched result field.
7641 - egrep mode now shows all hits rather than limiting the results to
7642 the top 20 hits.
7643
7644 - The documentation on how to format commands for ``datalad run`` has
7645 been improved.
7646 (`#2703 <https://github.com/datalad/datalad/issues/2703>`__)
7647 - The method for determining the current working directory on Windows
7648 has been improved.
7649 (`#2707 <https://github.com/datalad/datalad/issues/2707>`__)
7650 - ``datalad --version`` now simply shows the version without the
7651 license. (`#2733 <https://github.com/datalad/datalad/issues/2733>`__)
7652 - ``datalad export-archive`` learned to export under an existing
7653 directory via its ``--filename`` option.
7654 (`#2723 <https://github.com/datalad/datalad/issues/2723>`__)
7655 - ``datalad export-to-figshare`` now generates the zip archive in the
7656 root of the dataset unless ``--filename`` is specified.
7657 (`#2723 <https://github.com/datalad/datalad/issues/2723>`__)
7658 - After importing ``datalad.api``, ``help(datalad.api)`` (or
7659 ``datalad.api?`` in IPython) now shows a summary of the available
7660 DataLad commands.
7661 (`#2728 <https://github.com/datalad/datalad/issues/2728>`__)
7662 - Support for using ``datalad`` from IPython has been improved.
7663 (`#2722 <https://github.com/datalad/datalad/issues/2722>`__)
7664 - ``datalad wtf`` now returns structured data and reports the version
7665 of each extension.
7666 (`#2741 <https://github.com/datalad/datalad/issues/2741>`__)
7667 - The internal handling of gitattributes information has been improved.
7668 A user-visible consequence is that ``datalad create --force`` no
7669 longer duplicates existing attributes.
7670 (`#2744 <https://github.com/datalad/datalad/issues/2744>`__)
7671 - The “annex” metadata extractor can now be used even when no content
7672 is present.
7673 (`#2724 <https://github.com/datalad/datalad/issues/2724>`__)
7674 - The ``add_url_to_file`` method (called by commands like
7675 ``datalad download-url`` and ``datalad add-archive-content``)
7676 learned how to display a progress bar.
7677 (`#2738 <https://github.com/datalad/datalad/issues/2738>`__)
7678
7679 0.10.2 (Jul 09, 2018) – Thesecuriestever
7680 ========================================
7681
7682 Primarily a bugfix release to accommodate recent git-annex release
7683 forbidding file:// and http://localhost/ URLs which might lead to
7684 revealing private files if annex is publicly shared.
7685
7686 .. _fixes-36:
7687
7688 Fixes
7689 -----
7690
7691 - fixed testing to be compatible with recent git-annex (6.20180626)
7692 - `download-url <https://datalad.readthedocs.io/en/latest/generated/man/datalad-download-url.html>`__
7693 will now download to current directory instead of the top of the
7694 dataset
7695
7696 .. _enhancements-and-new-features-40:
7697
7698 Enhancements and new features
7699 -----------------------------
7700
7701 - do not quote ~ in URLs to be consistent with quote implementation in
7702 Python 3.7 which now follows RFC 3986
7703 - `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
7704 support for user-configured placeholder values
7705 - documentation on native git-annex metadata support
7706 - handle 401 errors from LORIS tokens
7707 - ``yoda`` procedure will instantiate ``README.md``
7708 - ``--discover`` option added to
7709 `run-procedure <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run-procedure.html>`__
7710 to list available procedures
7711
7712 0.10.1 (Jun 17, 2018) – OHBM polish
7713 ===================================
7714
7715 The is a minor bugfix release.
7716
7717 .. _fixes-37:
7718
7719 Fixes
7720 -----
7721
7722 - Be able to use backports.lzma as a drop-in replacement for pyliblzma.
7723 - Give help when not specifying a procedure name in ``run-procedure``.
7724 - Abort early when a downloader received no filename.
7725 - Avoid ``rerun`` error when trying to unlock non-available files.
7726
7727 0.10.0 (Jun 09, 2018) – The Release
7728 ===================================
7729
7730 This release is a major leap forward in metadata support.
7731
7732 .. _major-refactoring-and-deprecations-12:
7733
7734 Major refactoring and deprecations
7735 ----------------------------------
7736
7737 - Metadata
7738
7739 - Prior metadata provided by datasets under ``.datalad/meta`` is no
7740 longer used or supported. Metadata must be reaggregated using 0.10
7741 version
7742 - Metadata extractor types are no longer auto-guessed and must be
7743 explicitly specified in ``datalad.metadata.nativetype`` config
7744 (could contain multiple values)
7745 - Metadata aggregation of a dataset hierarchy no longer updates all
7746 datasets in the tree with new metadata. Instead, only the target
7747 dataset is updated. This behavior can be changed via the
7748 –update-mode switch. The new default prevents needless
7749 modification of (3rd-party) subdatasets.
7750 - Neuroimaging metadata support has been moved into a dedicated
7751 extension: https://github.com/datalad/datalad-neuroimaging
7752
7753 - Crawler
7754
7755 - moved into a dedicated extension:
7756 https://github.com/datalad/datalad-crawler
7757
7758 - ``export_tarball`` plugin has been generalized to ``export_archive``
7759 and can now also generate ZIP archives.
7760 - By default a dataset X is now only considered to be a super-dataset
7761 of another dataset Y, if Y is also a registered subdataset of X.
7762
7763 .. _fixes-38:
7764
7765 Fixes
7766 -----
7767
7768 A number of fixes did not make it into the 0.9.x series:
7769
7770 - Dynamic configuration overrides via the ``-c`` option were not in
7771 effect.
7772 - ``save`` is now more robust with respect to invocation in
7773 subdirectories of a dataset.
7774 - ``unlock`` now reports correct paths when running in a dataset
7775 subdirectory.
7776 - ``get`` is more robust to path that contain symbolic links.
7777 - symlinks to subdatasets of a dataset are now correctly treated as a
7778 symlink, and not as a subdataset
7779 - ``add`` now correctly saves staged subdataset additions.
7780 - Running ``datalad save`` in a dataset no longer adds untracked
7781 content to the dataset. In order to add content a path has to be
7782 given, e.g. ``datalad save .``
7783 - ``wtf`` now works reliably with a DataLad that wasn’t installed from
7784 Git (but, e.g., via pip)
7785 - More robust URL handling in ``simple_with_archives`` crawler
7786 pipeline.
7787
7788 .. _enhancements-and-new-features-41:
7789
7790 Enhancements and new features
7791 -----------------------------
7792
7793 - Support for DataLad extension that can contribute API components from
7794 3rd-party sources, incl. commands, metadata extractors, and test case
7795 implementations. See
7796 https://github.com/datalad/datalad-extension-template for a demo
7797 extension.
7798 - Metadata (everything has changed!)
7799
7800 - Metadata extraction and aggregation is now supported for datasets
7801 and individual files.
7802 - Metadata query via ``search`` can now discover individual files.
7803 - Extracted metadata can now be stored in XZ compressed files, is
7804 optionally annexed (when exceeding a configurable size threshold),
7805 and obtained on demand (new configuration option
7806 ``datalad.metadata.create-aggregate-annex-limit``).
7807 - Status and availability of aggregated metadata can now be reported
7808 via ``metadata --get-aggregates``
7809 - New configuration option ``datalad.metadata.maxfieldsize`` to
7810 exclude too large metadata fields from aggregation.
7811 - The type of metadata is no longer guessed during metadata
7812 extraction. A new configuration option
7813 ``datalad.metadata.nativetype`` was introduced to enable one or
7814 more particular metadata extractors for a dataset.
7815 - New configuration option
7816 ``datalad.metadata.store-aggregate-content`` to enable the storage
7817 of aggregated metadata for dataset content (i.e. file-based
7818 metadata) in contrast to just metadata describing a dataset as a
7819 whole.
7820
7821 - ``search`` was completely reimplemented. It offers three different
7822 modes now:
7823
7824 - ‘egrep’ (default): expression matching in a plain string version
7825 of metadata
7826 - ‘textblob’: search a text version of all metadata using a fully
7827 featured query language (fast indexing, good for keyword search)
7828 - ‘autofield’: search an auto-generated index that preserves
7829 individual fields of metadata that can be represented in a tabular
7830 structure (substantial indexing cost, enables the most detailed
7831 queries of all modes)
7832
7833 - New extensions:
7834
7835 - `addurls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-addurls.html>`__,
7836 an extension for creating a dataset (and possibly subdatasets)
7837 from a list of URLs.
7838 - export_to_figshare
7839 - extract_metadata
7840
7841 - add_readme makes use of available metadata
7842 - By default the wtf extension now hides sensitive information, which
7843 can be included in the output by passing ``--senstive=some`` or
7844 ``--senstive=all``.
7845 - Reduced startup latency by only importing commands necessary for a
7846 particular command line call.
7847 - `create <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create.html>`__:
7848
7849 - ``-d <parent> --nosave`` now registers subdatasets, when possible.
7850 - ``--fake-dates`` configures dataset to use fake-dates
7851
7852 - `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
7853 now provides a way for the caller to save the result when a command
7854 has a non-zero exit status.
7855 - ``datalad rerun`` now has a ``--script`` option that can be used to
7856 extract previous commands into a file.
7857 - A DataLad Singularity container is now available on `Singularity
7858 Hub <https://singularity-hub.org/collections/667>`__.
7859 - More casts have been embedded in the `use case section of the
7860 documentation <http://docs.datalad.org/en/docs/usecases/index.html>`__.
7861 - ``datalad --report-status`` has a new value ‘all’ that can be used to
7862 temporarily re-enable reporting that was disable by configuration
7863 settings.
7864
7865 0.9.3 (Mar 16, 2018) – pi+0.02 release
7866 ======================================
7867
7868 Some important bug fixes which should improve usability
7869
7870 .. _fixes-39:
7871
7872 Fixes
7873 -----
7874
7875 - ``datalad-archives`` special remote now will lock on acquiring or
7876 extracting an archive - this allows for it to be used with -J flag
7877 for parallel operation
7878 - relax introduced in 0.9.2 demand on git being configured for datalad
7879 operation - now we will just issue a warning
7880 - ``datalad ls`` should now list “authored date” and work also for
7881 datasets in detached HEAD mode
7882 - ``datalad save`` will now save original file as well, if file was
7883 “git mv”ed, so you can now ``datalad run git mv old new`` and have
7884 changes recorded
7885
7886 .. _enhancements-and-new-features-42:
7887
7888 Enhancements and new features
7889 -----------------------------
7890
7891 - ``--jobs`` argument now could take ``auto`` value which would decide
7892 on # of jobs depending on the # of available CPUs. ``git-annex`` >
7893 6.20180314 is recommended to avoid regression with -J.
7894 - memoize calls to ``RI`` meta-constructor – should speed up operation
7895 a bit
7896 - ``DATALAD_SEED`` environment variable could be used to seed Python
7897 RNG and provide reproducible UUIDs etc (useful for testing and demos)
7898
7899 0.9.2 (Mar 04, 2018) – it is (again) better than ever
7900 =====================================================
7901
7902 Largely a bugfix release with a few enhancements.
7903
7904 .. _fixes-40:
7905
7906 Fixes
7907 -----
7908
7909 - Execution of external commands (git) should not get stuck when lots
7910 of both stdout and stderr output, and should not loose remaining
7911 output in some cases
7912 - Config overrides provided in the command line (-c) should now be
7913 handled correctly
7914 - Consider more remotes (not just tracking one, which might be none)
7915 while installing subdatasets
7916 - Compatibility with git 2.16 with some changed behaviors/annotations
7917 for submodules
7918 - Fail ``remove`` if ``annex drop`` failed
7919 - Do not fail operating on files which start with dash (-)
7920 - URL unquote paths within S3, URLs and DataLad RIs (///)
7921 - In non-interactive mode fail if authentication/access fails
7922 - Web UI:
7923
7924 - refactored a little to fix incorrect listing of submodules in
7925 subdirectories
7926 - now auto-focuses on search edit box upon entering the page
7927
7928 - Assure that extracted from tarballs directories have executable bit
7929 set
7930
7931 .. _enhancements-and-new-features-43:
7932
7933 Enhancements and new features
7934 -----------------------------
7935
7936 - A log message and progress bar will now inform if a tarball to be
7937 downloaded while getting specific files (requires git-annex >
7938 6.20180206)
7939 - A dedicated ``datalad rerun`` command capable of rerunning entire
7940 sequences of previously ``run`` commands. **Reproducibility through
7941 VCS. Use ``run`` even if not interested in ``rerun``**
7942 - Alert the user if ``git`` is not yet configured but git operations
7943 are requested
7944 - Delay collection of previous ssh connections until it is actually
7945 needed. Also do not require ‘:’ while specifying ssh host
7946 - AutomagicIO: Added proxying of isfile, lzma.LZMAFile and io.open
7947 - Testing:
7948
7949 - added DATALAD_DATASETS_TOPURL=http://datasets-tests.datalad.org to
7950 run tests against another website to not obscure access stats
7951 - tests run against temporary HOME to avoid side-effects
7952 - better unit-testing of interactions with special remotes
7953
7954 - CONTRIBUTING.md describes how to setup and use ``git-hub`` tool to
7955 “attach” commits to an issue making it into a PR
7956 - DATALAD_USE_DEFAULT_GIT env variable could be used to cause DataLad
7957 to use default (not the one possibly bundled with git-annex) git
7958 - Be more robust while handling not supported requests by annex in
7959 special remotes
7960 - Use of ``swallow_logs`` in the code was refactored away – less
7961 mysteries now, just increase logging level
7962 - ``wtf`` plugin will report more information about environment,
7963 externals and the system
7964
7965 0.9.1 (Oct 01, 2017) – “DATALAD!”(JBTM)
7966 =======================================
7967
7968 Minor bugfix release
7969
7970 .. _fixes-41:
7971
7972 Fixes
7973 -----
7974
7975 - Should work correctly with subdatasets named as numbers of bool
7976 values (requires also GitPython >= 2.1.6)
7977 - Custom special remotes should work without crashing with git-annex >=
7978 6.20170924
7979
7980 0.9.0 (Sep 19, 2017) – isn’t it a lucky day even though not a Friday?
7981 =====================================================================
7982
7983 .. _major-refactoring-and-deprecations-13:
7984
7985 Major refactoring and deprecations
7986 ----------------------------------
7987
7988 - the ``files`` argument of
7989 `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
7990 has been renamed to ``path`` to be uniform with any other command
7991 - all major commands now implement more uniform API semantics and
7992 result reporting. Functionality for modification detection of dataset
7993 content has been completely replaced with a more efficient
7994 implementation
7995 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
7996 now features a ``--transfer-data`` switch that allows for a
7997 disambiguous specification of whether to publish data – independent
7998 of the selection which datasets to publish (which is done via their
7999 paths). Moreover,
8000 `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
8001 now transfers data before repository content is pushed.
8002
8003 .. _fixes-42:
8004
8005 Fixes
8006 -----
8007
8008 - `drop <http://datalad.readthedocs.io/en/latest/generated/man/datalad-drop.html>`__
8009 no longer errors when some subdatasets are not installed
8010 - `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
8011 will no longer report nothing when a Dataset instance was given as a
8012 source argument, but rather perform as expected
8013 - `remove <http://datalad.readthedocs.io/en/latest/generated/man/datalad-remove.html>`__
8014 doesn’t remove when some files of a dataset could not be dropped
8015 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
8016
8017 - no longer hides error during a repository push
8018 - publish behaves “correctly” for ``--since=`` in considering only
8019 the differences the last “pushed” state
8020 - data transfer handling while publishing with dependencies, to
8021 github
8022
8023 - improved robustness with broken Git configuration
8024 - `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
8025 should search for unicode strings correctly and not crash
8026 - robustify git-annex special remotes protocol handling to allow for
8027 spaces in the last argument
8028 - UI credentials interface should now allow to Ctrl-C the entry
8029 - should not fail while operating on submodules named with numerics
8030 only or by bool (true/false) names
8031 - crawl templates should not now override settings for ``largefiles``
8032 if specified in ``.gitattributes``
8033
8034 .. _enhancements-and-new-features-44:
8035
8036 Enhancements and new features
8037 -----------------------------
8038
8039 - **Exciting new feature**
8040 `run <http://datalad.readthedocs.io/en/latest/generated/man/datalad-run.html>`__
8041 command to protocol execution of an external command and rerun
8042 computation if desired. See
8043 `screencast <http://datalad.org/features.html#reproducible-science>`__
8044 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
8045 now uses Git for detecting with sundatasets need to be inspected for
8046 potential changes, instead of performing a complete traversal of a
8047 dataset tree
8048 - `add <http://datalad.readthedocs.io/en/latest/generated/man/datalad-add.html>`__
8049 looks for changes relative to the last committed state of a dataset
8050 to discover files to add more efficiently
8051 - `diff <http://datalad.readthedocs.io/en/latest/generated/man/datalad-diff.html>`__
8052 can now report untracked files in addition to modified files
8053 - [uninstall][] will check itself whether a subdataset is properly
8054 registered in a superdataset, even when no superdataset is given in a
8055 call
8056 - `subdatasets <http://datalad.readthedocs.io/en/latest/generated/man/datalad-subdatasets.html>`__
8057 can now configure subdatasets for exclusion from recursive
8058 installation (``datalad-recursiveinstall`` submodule configuration
8059 property)
8060 - precrafted pipelines of [crawl][] now will not override
8061 ``annex.largefiles`` setting if any was set within ``.gitattribues``
8062 (e.g. by ``datalad create --text-no-annex``)
8063 - framework for screencasts: ``tools/cast*`` tools and sample cast
8064 scripts under ``doc/casts`` which are published at
8065 `datalad.org/features.html <http://datalad.org/features.html>`__
8066 - new `project YouTube
8067 channel <https://www.youtube.com/channel/UCB8-Zf7D0DSzAsREoIt0Bvw>`__
8068 - tests failing in direct and/or v6 modes marked explicitly
8069
8070 0.8.1 (Aug 13, 2017) – the best birthday gift
8071 =============================================
8072
8073 Bugfixes
8074
8075 .. _fixes-43:
8076
8077 Fixes
8078 -----
8079
8080 - Do not attempt to
8081 `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
8082 a not installed sub-dataset
8083 - In case of too many files to be specified for
8084 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8085 or
8086 `copy_to <http://docs.datalad.org/en/latest/_modules/datalad/support/annexrepo.html?highlight=%22copy_to%22>`__,
8087 we will make multiple invocations of underlying git-annex command to
8088 not overfill command line
8089 - More robust handling of unicode output in terminals which might not
8090 support it
8091
8092 .. _enhancements-and-new-features-45:
8093
8094 Enhancements and new features
8095 -----------------------------
8096
8097 - Ship a copy of numpy.testing to facilitate [test][] without requiring
8098 numpy as dependency. Also allow to pass to command which test(s) to
8099 run
8100 - In
8101 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8102 and
8103 `copy_to <http://docs.datalad.org/en/latest/_modules/datalad/support/annexrepo.html?highlight=%22copy_to%22>`__
8104 provide actual original requested paths, not the ones we deduced need
8105 to be transferred, solely for knowing the total
8106
8107 0.8.0 (Jul 31, 2017) – it is better than ever
8108 =============================================
8109
8110 A variety of fixes and enhancements
8111
8112 .. _fixes-44:
8113
8114 Fixes
8115 -----
8116
8117 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
8118 would now push merged ``git-annex`` branch even if no other changes
8119 were done
8120 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
8121 should be able to publish using relative path within SSH URI (git
8122 hook would use relative paths)
8123 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
8124 should better tollerate publishing to pure git and ``git-annex``
8125 special remotes
8126
8127 .. _enhancements-and-new-features-46:
8128
8129 Enhancements and new features
8130 -----------------------------
8131
8132 - `plugin <http://datalad.readthedocs.io/en/latest/generated/man/datalad-plugin.html>`__
8133 mechanism came to replace
8134 `export <http://datalad.readthedocs.io/en/latest/generated/man/datalad-export.html>`__.
8135 See
8136 `export_tarball <http://docs.datalad.org/en/latest/generated/datalad.plugin.export_tarball.html>`__
8137 for the replacement of
8138 `export <http://datalad.readthedocs.io/en/latest/generated/man/datalad-export.html>`__.
8139 Now it should be easy to extend datalad’s interface with custom
8140 functionality to be invoked along with other commands.
8141 - Minimalistic coloring of the results rendering
8142 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__/``copy_to``
8143 got progress bar report now and support of ``--jobs``
8144 - minor fixes and enhancements to crawler (e.g. support of recursive
8145 removes)
8146
8147 0.7.0 (Jun 25, 2017) – when it works - it is quite awesome!
8148 ===========================================================
8149
8150 New features, refactorings, and bug fixes.
8151
8152 .. _major-refactoring-and-deprecations-14:
8153
8154 Major refactoring and deprecations
8155 ----------------------------------
8156
8157 - `add-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-add-sibling.html>`__
8158 has been fully replaced by the
8159 `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
8160 command
8161 - `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__,
8162 and
8163 `unlock <http://datalad.readthedocs.io/en/latest/generated/man/datalad-unlock.html>`__
8164 have been re-written to support the same common API as most other
8165 commands
8166
8167 .. _enhancements-and-new-features-47:
8168
8169 Enhancements and new features
8170 -----------------------------
8171
8172 - `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
8173 can now be used to query and configure a local repository by using
8174 the sibling name ``here``
8175 - `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
8176 can now query and set annex preferred content configuration. This
8177 includes ``wanted`` (as previously supported in other commands), and
8178 now also ``required``
8179 - New
8180 `metadata <http://datalad.readthedocs.io/en/latest/generated/man/datalad-metadata.html>`__
8181 command to interface with datasets/files
8182 `meta-data <http://docs.datalad.org/en/latest/cmdline.html#meta-data-handling>`__
8183 - Documentation for all commands is now built in a uniform fashion
8184 - Significant parts of the documentation of been updated
8185 - Instantiate GitPython’s Repo instances lazily
8186
8187 .. _fixes-45:
8188
8189 Fixes
8190 -----
8191
8192 - API documentation is now rendered properly as HTML, and is easier to
8193 browse by having more compact pages
8194 - Closed files left open on various occasions (Popen PIPEs, etc)
8195 - Restored basic (consumer mode of operation) compatibility with
8196 Windows OS
8197
8198 0.6.0 (Jun 14, 2017) – German perfectionism
8199 ===========================================
8200
8201 This release includes a **huge** refactoring to make code base and
8202 functionality more robust and flexible
8203
8204 - outputs from API commands could now be highly customized. See
8205 ``--output-format``, ``--report-status``, ``--report-type``, and
8206 ``--report-type`` options for
8207 `datalad <http://docs.datalad.org/en/latest/generated/man/datalad.html>`__
8208 command.
8209 - effort was made to refactor code base so that underlying functions
8210 behave as generators where possible
8211 - input paths/arguments analysis was redone for majority of the
8212 commands to provide unified behavior
8213
8214 .. _major-refactoring-and-deprecations-15:
8215
8216 Major refactoring and deprecations
8217 ----------------------------------
8218
8219 - ``add-sibling`` and ``rewrite-urls`` were refactored in favor of new
8220 `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
8221 command which should be used for siblings manipulations
8222 - ‘datalad.api.alwaysrender’ config setting/support is removed in favor
8223 of new outputs processing
8224
8225 .. _fixes-46:
8226
8227 Fixes
8228 -----
8229
8230 - Do not flush manually git index in pre-commit to avoid “Death by the
8231 Lock” issue
8232 - Deployed by
8233 `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
8234 ``post-update`` hook script now should be more robust (tolerate
8235 directory names with spaces, etc.)
8236 - A variety of fixes, see `list of pull requests and issues
8237 closed <https://github.com/datalad/datalad/milestone/41?closed=1>`__
8238 for more information
8239
8240 .. _enhancements-and-new-features-48:
8241
8242 Enhancements and new features
8243 -----------------------------
8244
8245 - new
8246 `annotate-paths <http://docs.datalad.org/en/latest/generated/man/datalad-annotate-paths.html>`__
8247 plumbing command to inspect and annotate provided paths. Use
8248 ``--modified`` to summarize changes between different points in the
8249 history
8250 - new
8251 `clone <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clone.html>`__
8252 plumbing command to provide a subset (install a single dataset from a
8253 URL) functionality of
8254 `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
8255 - new
8256 `diff <http://datalad.readthedocs.io/en/latest/generated/man/datalad-diff.html>`__
8257 plumbing command
8258 - new
8259 `siblings <http://datalad.readthedocs.io/en/latest/generated/man/datalad-siblings.html>`__
8260 command to list or manipulate siblings
8261 - new
8262 `subdatasets <http://datalad.readthedocs.io/en/latest/generated/man/datalad-subdatasets.html>`__
8263 command to list subdatasets and their properties
8264 - `drop <http://datalad.readthedocs.io/en/latest/generated/man/datalad-drop.html>`__
8265 and
8266 `remove <http://datalad.readthedocs.io/en/latest/generated/man/datalad-remove.html>`__
8267 commands were refactored
8268 - ``benchmarks/`` collection of `Airspeed
8269 velocity <https://github.com/spacetelescope/asv/>`__ benchmarks
8270 initiated. See reports at http://datalad.github.io/datalad/
8271 - crawler would try to download a new url multiple times increasing
8272 delay between attempts. Helps to resolve problems with extended
8273 crawls of Amazon S3
8274 - `CRCNS <http://crcns.org>`__ crawler pipeline now also fetches and
8275 aggregates meta-data for the datasets from datacite
8276 - overall optimisations to benefit from the aforementioned refactoring
8277 and improve user-experience
8278 - a few stub and not (yet) implemented commands (e.g. ``move``) were
8279 removed from the interface
8280 - Web frontend got proper coloring for the breadcrumbs and some
8281 additional caching to speed up interactions. See
8282 http://datasets.datalad.org
8283 - Small improvements to the online documentation. See e.g. `summary of
8284 differences between
8285 git/git-annex/datalad <http://docs.datalad.org/en/latest/related.html#git-git-annex-datalad>`__
8286
8287 0.5.1 (Mar 25, 2017) – cannot stop the progress
8288 ===============================================
8289
8290 A bugfix release
8291
8292 .. _fixes-47:
8293
8294 Fixes
8295 -----
8296
8297 - `add <http://datalad.readthedocs.io/en/latest/generated/man/datalad-add.html>`__
8298 was forcing addition of files to annex regardless of settings in
8299 ``.gitattributes``. Now that decision is left to annex by default
8300 - ``tools/testing/run_doc_examples`` used to run doc examples as tests,
8301 fixed up to provide status per each example and not fail at once
8302 - ``doc/examples``
8303
8304 - `3rdparty_analysis_workflow.sh <http://docs.datalad.org/en/latest/generated/examples/3rdparty_analysis_workflow.html>`__
8305 was fixed up to reflect changes in the API of 0.5.0.
8306
8307 - progress bars
8308
8309 - should no longer crash **datalad** and report correct sizes and
8310 speeds
8311 - should provide progress reports while using Python 3.x
8312
8313 .. _enhancements-and-new-features-49:
8314
8315 Enhancements and new features
8316 -----------------------------
8317
8318 - ``doc/examples``
8319
8320 - `nipype_workshop_dataset.sh <http://docs.datalad.org/en/latest/generated/examples/nipype_workshop_dataset.html>`__
8321 new example to demonstrate how new super- and sub- datasets were
8322 established as a part of our datasets collection
8323
8324 0.5.0 (Mar 20, 2017) – it’s huge
8325 ================================
8326
8327 This release includes an avalanche of bug fixes, enhancements, and
8328 additions which at large should stay consistent with previous behavior
8329 but provide better functioning. Lots of code was refactored to provide
8330 more consistent code-base, and some API breakage has happened. Further
8331 work is ongoing to standardize output and results reporting
8332 (`#1350 <https://github.com/datalad/datalad/issues/1350>`__)
8333
8334 Most notable changes
8335 --------------------
8336
8337 - requires `git-annex <http://git-annex.branchable.com/>`__ >=
8338 6.20161210 (or better even >= 6.20161210 for improved functionality)
8339 - commands should now operate on paths specified (if any), without
8340 causing side-effects on other dirty/staged files
8341 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
8342
8343 - ``-a`` is deprecated in favor of ``-u`` or ``--all-updates`` so
8344 only changes known components get saved, and no new files
8345 automagically added
8346 - ``-S`` does no longer store the originating dataset in its commit
8347 message
8348
8349 - `add <http://datalad.readthedocs.io/en/latest/generated/man/datalad-add.html>`__
8350
8351 - can specify commit/save message with ``-m``
8352
8353 - `add-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-add-sibling.html>`__
8354 and
8355 `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
8356
8357 - now take the name of the sibling (remote) as a ``-s`` (``--name``)
8358 option, not a positional argument
8359 - ``--publish-depends`` to setup publishing data and code to
8360 multiple repositories (e.g. github + webserve) should now be
8361 functional see `this
8362 comment <https://github.com/datalad/datalad/issues/335#issuecomment-277240733>`__
8363 - got ``--publish-by-default`` to specify what refs should be
8364 published by default
8365 - got ``--annex-wanted``, ``--annex-groupwanted`` and
8366 ``--annex-group`` settings which would be used to instruct annex
8367 about preferred content.
8368 `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
8369 then will publish data using those settings if ``wanted`` is set.
8370 - got ``--inherit`` option to automagically figure out url/wanted
8371 and other git/annex settings for new remote sub-dataset to be
8372 constructed
8373
8374 - `publish <http://datalad.readthedocs.io/en/latest/generated/man/datalad-publish.html>`__
8375
8376 - got ``--skip-failing`` refactored into ``--missing`` option which
8377 could use new feature of
8378 `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
8379 ``--inherit``
8380
8381 .. _fixes-48:
8382
8383 Fixes
8384 -----
8385
8386 - More consistent interaction through ssh - all ssh connections go
8387 through
8388 `sshrun <http://datalad.readthedocs.io/en/latest/generated/man/datalad-sshrun.html>`__
8389 shim for a “single point of authentication”, etc.
8390 - More robust
8391 `ls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-ls.html>`__
8392 operation outside of the datasets
8393 - A number of fixes for direct and v6 mode of annex
8394
8395 .. _enhancements-and-new-features-50:
8396
8397 Enhancements and new features
8398 -----------------------------
8399
8400 - New
8401 `drop <http://datalad.readthedocs.io/en/latest/generated/man/datalad-drop.html>`__
8402 and
8403 `remove <http://datalad.readthedocs.io/en/latest/generated/man/datalad-remove.html>`__
8404 commands
8405 - `clean <http://datalad.readthedocs.io/en/latest/generated/man/datalad-clean.html>`__
8406
8407 - got ``--what`` to specify explicitly what cleaning steps to
8408 perform and now could be invoked with ``-r``
8409
8410 - ``datalad`` and ``git-annex-remote*`` scripts now do not use
8411 setuptools entry points mechanism and rely on simple import to
8412 shorten start up time
8413 - `Dataset <http://docs.datalad.org/en/latest/generated/datalad.api.Dataset.html>`__
8414 is also now using `Flyweight
8415 pattern <https://en.wikipedia.org/wiki/Flyweight_pattern>`__, so the
8416 same instance is reused for the same dataset
8417 - progressbars should not add more empty lines
8418
8419 Internal refactoring
8420 --------------------
8421
8422 - Majority of the commands now go through ``_prep`` for arguments
8423 validation and pre-processing to avoid recursive invocations
8424
8425 0.4.1 (Nov 10, 2016) – CA release
8426 =================================
8427
8428 Requires now GitPython >= 2.1.0
8429
8430 .. _fixes-49:
8431
8432 Fixes
8433 -----
8434
8435 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
8436
8437 - to not save staged files if explicit paths were provided
8438
8439 - improved (but not yet complete) support for direct mode
8440 - `update <http://datalad.readthedocs.io/en/latest/generated/man/datalad-update.html>`__
8441 to not crash if some sub-datasets are not installed
8442 - do not log calls to ``git config`` to avoid leakage of possibly
8443 sensitive settings to the logs
8444
8445 .. _enhancements-and-new-features-51:
8446
8447 Enhancements and new features
8448 -----------------------------
8449
8450 - New `rfc822-compliant
8451 metadata <http://docs.datalad.org/en/latest/metadata.html#rfc822-compliant-meta-data>`__
8452 format
8453 - `save <http://datalad.readthedocs.io/en/latest/generated/man/datalad-save.html>`__
8454
8455 - -S to save the change also within all super-datasets
8456
8457 - `add <http://datalad.readthedocs.io/en/latest/generated/man/datalad-add.html>`__
8458 now has progress-bar reporting
8459 - `create-sibling-github <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling-github.html>`__
8460 to create a :term:``sibling`` of a dataset on github
8461 - `OpenfMRI <http://openfmri.org>`__ crawler and datasets were enriched
8462 with URLs to separate files where also available from openfmri s3
8463 bucket (if upgrading your datalad datasets, you might need to run
8464 ``git annex enableremote datalad`` to make them available)
8465 - various enhancements to log messages
8466 - web interface
8467
8468 - populates “install” box first thus making UX better over slower
8469 connections
8470
8471 0.4 (Oct 22, 2016) – Paris is waiting
8472 =====================================
8473
8474 Primarily it is a bugfix release but because of significant refactoring
8475 of the
8476 `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
8477 and
8478 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8479 implementation, it gets a new minor release.
8480
8481 .. _fixes-50:
8482
8483 Fixes
8484 -----
8485
8486 - be able to
8487 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8488 or
8489 `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
8490 while providing paths while being outside of a dataset
8491 - remote annex datasets get properly initialized
8492 - robust detection of outdated
8493 `git-annex <http://git-annex.branchable.com/>`__
8494
8495 .. _enhancements-and-new-features-52:
8496
8497 Enhancements and new features
8498 -----------------------------
8499
8500 - interface changes
8501
8502 - `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8503 ``--recursion-limit=existing`` to not recurse into not-installed
8504 subdatasets
8505 - `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8506 ``-n`` to possibly install sub-datasets without getting any data
8507 - `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
8508 ``--jobs|-J`` to specify number of parallel jobs for annex
8509 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8510 call could use (ATM would not work when data comes from archives)
8511
8512 - more (unit-)testing
8513 - documentation: see http://docs.datalad.org/en/latest/basics.html for
8514 basic principles and useful shortcuts in referring to datasets
8515 - various webface improvements: breadcrumb paths, instructions how to
8516 install dataset, show version from the tags, etc.
8517
8518 0.3.1 (Oct 1, 2016) – what a wonderful week
8519 ===========================================
8520
8521 Primarily bugfixes but also a number of enhancements and core
8522 refactorings
8523
8524 .. _fixes-51:
8525
8526 Fixes
8527 -----
8528
8529 - do not build manpages and examples during installation to avoid
8530 problems with possibly previously outdated dependencies
8531 - `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
8532 can be called on already installed dataset (with ``-r`` or ``-g``)
8533
8534 .. _enhancements-and-new-features-53:
8535
8536 Enhancements and new features
8537 -----------------------------
8538
8539 - complete overhaul of datalad configuration settings handling (see
8540 `Configuration
8541 documentation <http://docs.datalad.org/config.html>`__), so majority
8542 of the environment. Now uses git format and stores persistent
8543 configuration settings under ``.datalad/config`` and local within
8544 ``.git/config`` variables we have used were renamed to match
8545 configuration names
8546 - `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__
8547 does not now by default upload web front-end
8548 - `export <http://datalad.readthedocs.io/en/latest/generated/man/datalad-export.html>`__
8549 command with a plug-in interface and ``tarball`` plugin to export
8550 datasets
8551 - in Python, ``.api`` functions with rendering of results in command
8552 line got a \_-suffixed sibling, which would render results as well in
8553 Python as well (e.g., using ``search_`` instead of ``search`` would
8554 also render results, not only output them back as Python objects)
8555 - `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8556
8557 - ``--jobs`` option (passed to ``annex get``) for parallel downloads
8558 - total and per-download (with git-annex >= 6.20160923) progress
8559 bars (note that if content is to be obtained from an archive, no
8560 progress will be reported yet)
8561
8562 - `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
8563 ``--reckless`` mode option
8564 - `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
8565
8566 - highlights locations and fieldmaps for better readability
8567 - supports ``-d^`` or ``-d///`` to point to top-most or centrally
8568 installed meta-datasets
8569 - “complete” paths to the datasets are reported now
8570 - ``-s`` option to specify which fields (only) to search
8571
8572 - various enhancements and small fixes to
8573 `meta-data <http://docs.datalad.org/en/latest/cmdline.html#meta-data-handling>`__
8574 handling,
8575 `ls <http://datalad.readthedocs.io/en/latest/generated/man/datalad-ls.html>`__,
8576 custom remotes, code-base formatting, downloaders, etc
8577 - completely switched to ``tqdm`` library (``progressbar`` is no longer
8578 used/supported)
8579
8580 0.3 (Sep 23, 2016) – winter is coming
8581 =====================================
8582
8583 Lots of everything, including but not limited to
8584
8585 - enhanced index viewer, as the one on http://datasets.datalad.org
8586 - initial new data providers support:
8587 `Kaggle <https://www.kaggle.com>`__,
8588 `BALSA <http://balsa.wustl.edu>`__,
8589 `NDA <http://data-archive.nimh.nih.gov>`__,
8590 `NITRC <https://www.nitrc.org>`__
8591 - initial `meta-data support and
8592 management <http://docs.datalad.org/en/latest/cmdline.html#meta-data-handling>`__
8593 - new and/or improved crawler pipelines for
8594 `BALSA <http://balsa.wustl.edu>`__, `CRCNS <http://crcns.org>`__,
8595 `OpenfMRI <http://openfmri.org>`__
8596 - refactored
8597 `install <http://datalad.readthedocs.io/en/latest/generated/man/datalad-install.html>`__
8598 command, now with separate
8599 `get <http://datalad.readthedocs.io/en/latest/generated/man/datalad-get.html>`__
8600 - some other commands renaming/refactoring (e.g.,
8601 `create-sibling <http://datalad.readthedocs.io/en/latest/generated/man/datalad-create-sibling.html>`__)
8602 - datalad
8603 `search <http://datalad.readthedocs.io/en/latest/generated/man/datalad-search.html>`__
8604 would give you an option to install datalad’s super-dataset under
8605 ~/datalad if ran outside of a dataset
8606
8607 0.2.3 (Jun 28, 2016) – busy OHBM
8608 --------------------------------
8609
8610 New features and bugfix release
8611
8612 - support of /// urls to point to http://datasets.datalad.org
8613 - variety of fixes and enhancements throughout
8614
8615 0.2.2 (Jun 20, 2016) – OHBM we are coming!
8616 ------------------------------------------
8617
8618 New feature and bugfix release
8619
8620 - greatly improved documentation
8621 - publish command API RFing allows for custom options to annex, and
8622 uses –to REMOTE for consistent with annex invocation
8623 - variety of fixes and enhancements throughout
8624
8625 0.2.1 (Jun 10, 2016)
8626 --------------------
8627
8628 - variety of fixes and enhancements throughout
8629
8630 0.2 (May 20, 2016)
8631 ==================
8632
8633 Major RFing to switch from relying on rdf to git native submodules etc
8634
8635 0.1 (Oct 14, 2015)
8636 ==================
8637
8638 Release primarily focusing on interface functionality including initial
8639 publishing