With 37,000 employees, the FBI’s HR system has long been challenged to meet intense demands for reporting. They carried an existing sitewide license of MicroStrategy, Informatica, and Oracle Database that SMX improved upon to build an HR Data Warehouse and deliver enhanced reporting for personnel management functions, including talent acquisition and performance. Often, multiple ad-hoc requests would conflict with pre-assigned deliverables, and the Bureau’s home-built legacy system, BPMS (Bureau Personnel Management System) lacked the task management capabilities needed to support the volume of inquiries.


The solution was to replace the legacy, home-built personnel management system with a streamlined, modern platform for all HR Information within one system.

SMX migrated the FBI’s architecture to Amazon Redshift, a cloud-based data warehouse that can analyze exabytes of data simultaneously. By migrating to Amazon Redshift, the speed of data warehouse queries increased by 3x or more and offered a seamless integration with S3 allowing the customer to parse storage from compute. This Extract-Load-Transform (ELT) platform, branded as Datashift, increased the number of complex queries answered using the FBI’s HR data.

Datashift uses Amazon Database Migration Service (DMS) for real-time change-data-capture updates, Apache Airflow for job orchestration, and the high-powered Amazon Redshift database allowing for the ELT process to run continuously throughout the business day. This end-to-end process gave the customer’s Human Resources division access to near real-time data compared to the data that was only available to users from the nightly batch loads on the previous platform.

Our team leveraged agile methods using 2-week sprints; priorities were set during weekly collaborative meetings with the client. As a result, end users experienced improved ease of use and enhanced interaction, gaining the ability to produce robust reports that are more specific to business needs.


SMX has helped the HR division collect, maintain, and utilize data more effectively resulting in the reduction of staffing gaps with predictive hiring, improvement in diversity hiring and assessing risks of insider threats.

Contact Us

Wherever you are on your transformation journey, SMX can help you get there faster and more effectively.

Contact us to learn more.