this is to improve usage of s3 data grabber.
several examples here: https://.com/kellyhennigan/cueexp_scripts/blob/25372d4f221762af6d79d1af67a8570f2c546b20/svmrfe/venv/lib64/python2.7/site-packages/nipype/interfaces/tests/test_io.py

Acknowledgment

• (Mandatory) I acknowledge that this contribution will be available under the Apache 2 license.

Codecov Report

Merging #2849 into master will decrease coverage by 4.58%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2849      +/-   ##
==========================================
- Coverage   67.47%   62.88%   -4.59%     
==========================================
  Files         341      339       -2     
  Lines       43362    43311      -51     
  Branches     5379     5095     -284     
==========================================
- Hits        29257    27236    -2021     
- Misses      13409    15039    +1630     
- Partials      696     1036     +340

Flag	Coverage Δ
#smoketests	?
#unittests	62.88% <ø> (-2.01%)	⬇️

Impacted Files	Coverage Δ
nipype/interfaces/io.py	47.26% <ø> (-6.73%)	⬇️
nipype/interfaces/nilearn.py	40% <0%> (-56.67%)	⬇️
nipype/utils/spm_docs.py	25.92% <0%> (-44.45%)	⬇️
nipype/testing/fixtures.py	56% <0%> (-42.67%)	⬇️
nipype/interfaces/nitime/analysis.py	43.15% <0%> (-36.85%)	⬇️
nipype/interfaces/fsl/model.py	49.11% <0%> (-31.51%)	⬇️
nipype/interfaces/freesurfer/base.py	50% <0%> (-30.51%)	⬇️
nipype/utils/logger.py	59.7% <0%> (-29.86%)	⬇️
nipype/algorithms/rapidart.py	35% <0%> (-29.42%)	⬇️
nipype/interfaces/spm/base.py	58.08% <0%> (-29.05%)	⬇️
... and 54 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update add9bd7...ea4fb4b. Read the comment docs.

@TimVanMourik ( here is an example of s3 datagrabber )

PEP8 issues identified by Codacy.

Thanks for this example. The s3grab.inputs.template = '*' suggests a glob style wildcard, while the docstring claims (correctly) that for the S3DataGrabber a regex is used. So while this example works, maybe a more regex-looking template spec would highlight that difference from DataGrabber to the reader.

Going to go ahead and merge this. @wtriplett Feel free to open a pull request with an improved (or, better, another) example.

@TimVanMourik were you able to use this to create an s3datagrabber example? i'm not completely sure how i would initialize the s3datagrabber appropriately with giraffe.

@wtriplett - it would be nice to add a regex example. i just wanted to get an example in, since there were none :)

Sorry for the delay. Yes, made an example:
https://giraffe.tools/porcupine/TimVanMourik/SomeGiraffeExample/s3 or a permanent link to current status, in case the branch gets updated.

This example:

• It implements the file handling by:
  • mapping the input subj_id and run_num ...
  • ...via the field_template and template_args...
  • ...to output func and struct.
  • It's far from intuitive but it's the way the S3DataGrabber works so there's not much I can do about it from my side.
• Grabs data from the example file you mentioned
  • Iterates over subjects in the first node, but not over runs: the structural pipeline needs to be performed only for each subject. The functional pipeline is later also iterated over runs.
• Further implements a basic pipeline and stores the data at the end.

All Nipype modules that requires dynamic inputs/outputs are a bit cumbersome in GiraffeTools, but can be dealt with by adding ports manually. It is a bit contrary to the philosophy though, as I like to think of functions as static.