ML-based solution for recruitment automation

#AI
#Data analytics
#Machine learning
#Text analysis

Business Challenge

Comeet operates a collaborative recruitment platform with a goal to make the process of hiring personalized.

Usually, recruiters need somewhat between one week and a month to find suitable candidates for a position.

Our client wanted to automate finding candidates whose CVs match with vacancies.

This would save even small companies hundreds of hours annually.

Solution Overview

We created a smart algorithm for matching recruiters’ search demands with CVs that helps to find specialists from different areas, not just by keywords.

We also implemented another algorithm that shows how much a person fits a position and sorts of candidates in the order of suitability.

The search system lets users add additional filters or prioritize skills.

Project Description

To get data from CVs, we used a special parser that made the development much faster. Our algorithm works on two converted text lines from CVs and recruiters’ filters. We used ElasticSearch and modern NLP techniques to make the search as fast as possible.

The biggest challenge was deciding how to compare people with the same title in the profile (for example “Doctor”) but with different skill sets or vice versa. Also, the algorithm had to understand the slight differences between related fields and areas of specialization.

Our solution could roughly be split into three sub-solutions:

comparison of a job description and a profile
matching with extra filters
sorting by prioritizing skills, employment history, education history, etc.

We added 12 criteria to sort the final results. Each candidate got assigned points and the system calculated an average score.

Let's discuss your idea!

Technological Details

The technology stack the owner chose for this project included Python, ElasticSearch, ElasticSearch-DSL-Py, AWS S3 bucket, MySQL, AWS RedShift and Django REST framework.

The main goal was to develop a search REST API as a part of an existing project powered by Django REST Framework for the client app. The search API uses ElasticSearch and enables multi-criteria searching functionality.

This task required the following:

analyzing documents that will be saved to the ElasticSearch index, which stored at AWS s3 bucket, using AWS RedShift.
creating synonym tables and available values for search tables, which stored at MySQL.
creating and configuring the ElasticSearch index.
creating a REST API handler for search using the Django REST framework.
creating a handler for converting client application search requests into ElasticSearch queries using Python and ElasticSearch DSL for Python.
creating custom aggregation and search filter scripts in ElasticSearch.
creating a ranking algorithm using Python and custom ElasticSearch scripts.
creating an ETL task to synchronize AWS RedShift tables with AWS S3 Bucket.