About
This project focuses on the identification and analysis of mega projects by evaluating a large volume of associated invoices. The project's primary goal is to connect these mega projects to clients' invoices, offering insights into their overall performance and identifying potential opportunities for growth. To achieve these objectives, the project employs a comprehensive data analytics and machine learning approach.
Challenge, approach, and impact
Data Integration and Cross-Referencing
One of the major hurdles was integrating data from various third-party systems, including obtaining construction permits and location information. We needed to cross-reference this data with our internal records, which required careful validation to ensure accuracy.
Data Cleaning
The addresses in our databases were inconsistent and required significant cleaning. The process of standardizing and validating these addresses to accurately pinpoint coordinates was complex and time-consuming.
Geolocation Issues
Accurately converting cleaned addresses into geographic coordinates for proper cross-referencing was challenging, especially when dealing with incomplete or ambiguous location data.
Data Volume
The large volume of invoices and associated project data created processing challenges. Efficiently handling and analyzing such a vast dataset required advanced techniques and reliable infrastructure to ensure timely and accurate results.
Technology Integration
Using a combination of Azure, AWS, Python, SQL Server, and C# required ensuring smooth interoperability between these platforms, which added complexity to the project's technical execution.
How we built
Testimonials
Anonymous
Diligent Solutions DOO
“Working on this project has been a rewarding experience. By analyzing mega project invoices, we’ve gained valuable insights into performance and identified growth opportunities. Using technologies like Python, SQL Server, AWS, and Azure, we efficiently processed and analyzed large datasets, uncovering trends that drive key decisions. It’s been a great opportunity to work with cutting-edge tools and contribute to a project with a significant impact on future growth.“
Team structure
Client team
Brandon L
Vice President - Enterprise Data
Project stakeholder
The client stakeholders were working closely with the team at Diligent Solutions
Agency team
1 x Project Manager
Production
1 x Solution Architect
Production
3 x Data Scientist
Production
