Services

Data Collection, Cleaning, and Processing

Data Collection

Data can be found in many disparate and messy forms, and proper methods of extracting this data can increase efficiency and accuracy in the collection process. J&M specializes in the field known as data “scraping” or data “harvesting,” in which custom programs can be built to gather data organized in similarly formatted files. We have developed proprietary algorithms that readily obtain information from file types such as text documents, PDFs, e-mail chains, images, and websites. As a result, our methods condense data into useable storage applications, such as spreadsheets or databases.

Examples from select industries include:

Law: Paralegals or interns are often employed to sift through documents, contracts, or e-mail chains in search of dates, services, quantities, fees, and other standardized fields. Such repetitive human data entry is prone to mistakes and is time consuming, and we can build automated processes to collect this data with as little error as possible. In addition, documents are often scanned due to signatures or other handwritten notes, and we can extract data through optical character recognition (OCR) technology.

Finance: Many financial services firms require information from popular databases such as Bloomberg, but need ways to automate the extraction process for thousands of securities. While plugins exist for programs like Excel, more advanced data collection processes involving corporate databases and servers are typically needed. Firms find our data collection services especially useful because we can link company-specific data storage programs, external financial databases, and our data analysis procedures in a seamless way.

Real Estate: Information on real estate properties is often scattered and varies across regional markets. Our algorithms can efficiently gather data such as assessed and appraised values, sales data, and property specifics like size, location, building materials, and number and types of rooms.

Data Cleaning

Data collected in its raw form usually suffers from errors in entry, and we provide services that rectify these mistakes. Typically, websites or other searchable files are constructed a manual recording of the data at some level, and image files (such as PDF, JPEG, or TIFF) can be distorted during the text conversion process. We can perform quality checks of data provided to firms from third party vendors or gathered from our data collection methods. In doing so, our team combines logical and statistical tests with algorithms for efficient manual data checking to ensure complete accuracy.

Data Processing

While our data collection and cleaning services are valuable as static services, we recognize that ongoing procedures for newly available data are more beneficial to our clients. We provide such automated procedures, the necessary education to keep them functional, and future support in data cleaning. In addition, we can implement data and file management systems that allow for ease of use and accessibility of stored information.

Data Analysis, Model and Algorithm Building, and Third Party Quantitative Work

Data Analysis

J&M’s second specialty involves the proper analysis and evaluation of data. At its most basic level, such analysis could be an exploratory study, comprehensive testing of variables and their relationships, or graphical representations. More advanced techniques include identifying and isolating clusters of data, reducing variables for simpler interpretation, and extensive probability analysis. We understand that data is found in a variety of forms, and our goal is to allow clients to easily draw conclusions and glean as much information in keeping with correct statistical theory.

Model and Algorithm Building

Equally important to evaluating variables in their current state is creating models of prediction and ongoing algorithms. Our academic and professional experience gives us unique insight when developing dynamic and customized solutions for our clients. Some examples within this realm include prediction models for real estate markets or new financial product performance, and algorithms for matching individuals on both common and distinct characteristics.

Third Party Quantitative Work

In addition to providing independent analysis and model and algorithm building, J&M can serve as a third party quantitative team for projects or analyses typically performed by in-house quantitative or statistical departments. Through experience working for in-house teams, we realize that certain quantitative projects are important yet neglected in the face of personnel or time shortages. Our capabilities in performing third party quantitative work and working with in-house teams is an attractive alternative to hiring full-time employees.