TensorFlow Extended Addon
I pushed my first open source commit to TensforFlow Extended.
This component is called CopyExampleGen and will save developers significant time (hours for large amounts of data) per pipeline run when a pipeline run does not require data to be shuffled.
The component will accept TFRecord files and register them as an Examples Artifact for downstream components to use. CopyExampleGen accepts a dictionary where keys are the split-names and their respective value is a URI to the folder that contains the TFRecords file(s).
Worked alongside Spotify ML Engineers and other open source developers to learn more about their struggles with TensorFlow Extended. I developed a roadmap and purpose for this component, ensuring it solved the intended challenge while meeting all user’s requests.
I was able to roll this component out in less than 2 months. This component is now available as a TensorFlow Extended Addon component for open source use!
Me and the mentor.
