At Lazarus, we build foundation models to help people solve the world's toughest problems. Enterprise and government clients use our APIs to power products and workflows that keep humans focused on what we do best. We're a venture-backed team headquartered in the Boston, MA area, and we share a passion for innovation, growth, and a commitment to the responsible use of AI.
We are seeking a highly motivated and technically adept Data Engineer interested in contributing to our Machine Learning products in a hybrid role encompassing Testing and Data Engineering. The ideal candidate can take ownership of managing various data sets that we use for training, evaluation, and proving new capabilities while leveraging scripting and Python skills to implement and automate testing for our range of document processing capabilities. The data needs to be stored securely (per company policies) while providing uniform easy access to anyone within the Lazarus team who needs access. You will work closely with our QA and testing engineers as they seek to automate regression and other testing for formal product releases, ensuring that the data sets can evolve over time (via versioning) and managing both the data and any ground truth labels associated with the data and extending their utilization through the Model Development lifecycle. You will also bring best practices for data management to the team including building a data catalog so we always understand the data available and creating the foundation for an internal data mart supporting platform monitoring and a full range of business intelligence operations while building infrastructure for automating and scaling testing of capabilities.
Responsibilities
- Contribute to discussions and decisions around infrastructure options for data storage, both offline for training as well as for production system.
- Build a data catalog capturing what data we have, its description and usage, and any licensing status/issues/privacy concerns for specific data sets.
- Work with ML team and Implementation/customer-facing team(s) to add new data from customers, standard tests, open sources, and synthetic data.
- Version control data sets, including metadata and truth labels associated with data sets, track evolution of open source/external data.
- Apply standard normalization as well as pre-processing to data sets.
- Learn how to run our Synthetic Document Generation tool to construct or augment data sets.
- Develop and extend testing scripts and infrastructure to standardize and automate elements of testing and validation across our products.
- Build a pipeline to collect and generate near-real-time metrics around production performance.
Requirements
- BS in STEM field or significant experience
- 2+ years of experience Python coding in a development environment (proficient to expert in Python)
- Experience working with Google Cloud Platform
- Test scripting or test automation experience
Nice-to-haves
- REST interfaces and using Postman to exercise endpoints
- Pipeline implementation experience with tools like AirFlow or DBT
- SQL and NoSQL querying
- Experienced in building apps leveraging APIs
Lazarus AI is an equal opportunity employer. We are committed to equal employment opportunity and nondiscrimination for all employees and qualified applicants without regard to a person's race, color, gender, age, religion, national origin, ancestry, disability, veteran status, genetic information, sexual orientation or any characteristic protected under applicable law. We do not tolerate discrimination or harassment of any kind. This applies to every aspect of employment at Lazarus, including, but not limited to, employment, training, promotion, demotion, transfer, leaves of absence and termination.
Apply for this job
* indicates a required field
First Name *
Last Name *
Email *
Phone
Resume/CV *
Accepted file types: pdf, doc, docx, txt, rtf
LinkedIn Profile
Website
Are you authorized to work in the U.S.? * Select...
Will you now or in the future require sponsorship to work in the U.S.? * Select...
#J-18808-Ljbffr