Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configurability to custom Steps #915

Open
neolefty opened this issue May 29, 2024 · 2 comments
Open

Add configurability to custom Steps #915

neolefty opened this issue May 29, 2024 · 2 comments

Comments

@neolefty
Copy link

Overview

A research team within the Purdue College of Agriculture has started to use Frictionless for its data pipelines, and we would like to write custom Steps that receive configuration options from the Pipeline YAML file. For example:

steps:

  # Standard Step -- ensure ID is an integer
  - type: field-update
    name: id
    descriptor:
      name: farm_animal_id
      type: integer

  # Custom Step -- normalize funky date formats
  # Currently, in our testing, values (below) such as field_name or descriptor.name
  # are not getting passed to the custom step, and it appears they are not kept by
  # the custom Step mechanism. We want to capture them and pass them into the
  # custom Step initialization.
  - type: date-normalize
    field_name: Date
    descriptor:
      name: Date
      type: string
@neolefty
Copy link
Author

Hi Frictionless! We want to develop and submit a solution, and also plan to attend the hangout tomorrow, to get connected to the Frictionless community.

@neolefty
Copy link
Author

Question: Should this move over to frictionless-py? https://github.com/frictionlessdata/frictionless-py/issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant