I was on the Computational Modeling Engineering (CME) team at Montai. Computational Modeling (CM) was responsible for developing, building, and deploying ML models for the CONECTA platform with 2 teams: Science (CMS) who developed the models and Engineering (CME) who helped the scientists with various engineering tasks.
I helped support the CM Scientists in various ways: building/maintaining tools (in-house experiment tracking, testing, cloud deployment, workflow orchestration “with Redun”), managing environments, improving code quality & documentation, and many other small but consequential tasks.
The main skill I developed over time was understanding the needs of our team and finding simple and effective ways to solve them (low overhead, simple-to-use, gets the job done).
Results
- Enhanced machine learning scientists’ productivity by 80% & reduced errors by 50% by developing a tool to deploy models in AWS EC2 instances with adjustable hyperparameters, streamlining scientists’ workflows
- Accelerated Tensorboard data loading by 97% by engineering a tool that queries and launches with essential files, significantly enhancing data accessibility and analysis efficiency
- Optimized model training costs by incorporating EC2 Spot Instances with up to 90% reduced cost
- Sped up & automated end-to-end testing by 70% by parallelizing tests to run on different Amazon EC2 instances & environments (Anaconda)
- Built an ETL tool to seamlessly work with redun to speed up extract-transform-load (ETL) tasks by 50%
- Refactored an internal experiment tracking tool (4000 lines), improving speed of understanding and developing new features by 50%
- Led transition from GitHub Projects to Jira to better coordinate project management workflows of 3 teams (15 members) by conducting research, interviews, presentations, and integration.