[New WfMS] Flowrep

[jan-janssen]

pyiron_workflow is now using flowrep recipes as translation format. So ideally the Python Workflow Definition should also support flowrep. But we have not shown this explicitly for the three examples we discuss:
https://github.com/pyiron/flowrep

[liamhuber]

I'm working on this, and I've hit a roadblock. When I wrote the flowrep<->PWD converters I completely forgot that PWD exclusively supports functions with a single return value, and that all recipes with multiple output ports are implicitly expecting the function to return a dictionary with keys corresponding to port labels. This is just a fundamentally different assumption than flowrep, which assumes that separate output ports refer to separate return values.

That means that while the recipes round-trip recipe<->other recipe format<->recipe just fine, if you try to actually execute one of those recipes in a WfMS, and any of the atomic/function nodes is keyed for multiple-output, a flowrep-oriented WfMS running on top of PWD-oriented atomic function will complain because it's trying to unpack a single return value (the dictionary) into multiple reports; similarly, executing a PWD-oriented WfMS on top of a flowrep-oriented function will complain because the WfMS has no idea what to do with a tuple return value and is freaked out because there are no dictionary keys to match ports to.

For the case of flat DAGs (PWD constraint) where all atomic nodes have a single return port (where interpretation assumptions overlap), everything is still fine.

The generic brute-force solution for PWD->flowrep is to convert multi-output nodes into single-output atomic nodes and then inject additional dictionary key access nodes to split the output into multiple ports; this is annoying, but doable. For the reverse process, each flowrep atomic node with multiple outputs converts to a single-output PWD recipe and then needs to pass the output there to a new injected node that converts those inputs to a dictionary. I don't immediately see a trivial way to do this, but something similar is already done here for the old pyiron_workflow support that involves execing on-the-fly function strings -- not my favourite.

All this is more work than I had anticipated, and I'm rather crunched right now so I'm afraid I can't give a timeline for it. What I will do is go patch flowrep so that it fails cleanly in the multiple-outputs case, so at least we'll be completely honest about what can and can't round-trip convert. Honestly, I'd rather not have to do the fixes at all and just have PWD adopt flowrep's interpretation of atomic nodes -- force-breaking dictionary returns into output ports is a bit wonky, but much more seriously it's extremely restrictive as there are many existing python functions with multiple return values and, as far as I can tell, these are all precluded by the current PWD paradigm, rather requiring and explicit "to a dict-returner" wrapper.