Alegion - Computer Vision

Summary

Alegion was always pushing the edge of exploring how we could use machine learning and computer vision to reduce the time it took to label complex images. That was a huge part of my job. This project spanned my entire time at Alegion. I continuously worked with our computer vision researchers and backend engineers to create high fidelity prototypes that explored how our data labelers would used various ML and CV techniques to reduce the amount of repetitive and/or unnecessary work our data labelers needed to do.

The end result of this project resulted in one of my prototypes being added to our production project to help with a complex data labeling client. The project helped cut cost by approximately $100,000 and reduce the time to label per image by 80%.

‍

Technologies Used

1. TypeScript

2. React

3. Material UI

4. Styled Components

5. Python

6. HTML5 Canvas

‍

Design Considerations

This project needed to be built in such a way that it was easy to plugin different computer visions algorithms that users could then use to label an example image and provide feedback on whether or not this may their job easier or more difficult.

‍

Architecture

This app was built to dynamically plugin any computer vision algorithm by creating a dynamic plugin system that adhered to a strict set of protocols for any algorithm that was to be imported. Each algorithm was extended a base model class that required the algorithm to implement certain methods and tell react what parameters this algorithm needed in order for the user to use it. For example, if one of the parameters was region size, which we wanted rendered as a range, we would specify that using our model and then react would handle rendering the components and make the appropriate api calls to get back the image data.

The image data would then be rendered onto the canvas and the user would be able to interact with it using various tools such as a polygon, paint brush, eraser etc... Each algorithm also specified what tools a user should be able to use and could customize each tool by extending our Tools model.

‍