Debian Codegraph @

releasenotes/notes/wf_final_context_evaluation_with_batches-6292ab64c131dfcc.yaml

raw
1 ---
2 fixes:
3 - Evaluation of final workflow context was very heavy in cases
4 when the workflow had a lot of parallel tasks with large
5 inbound contexts. Merging of those contexts in order to
6 evaluate the workflow output consumed a lot of memory.
7 Now this algorithm is rewritten with batched DB query and
8 Python generators so that GS has a chance to destroy
9 objects that have already been processed. Previously all
10 task executions had to stay in memory until the end of the
11 processing. The result is that now it consumes 3 times less
12 memory on heavy cases.