One of the biggest challenges when developing or testing a model is accessing the right data. In some cases, it can take months or even years to get access to sensitive data sets.
In such cases, synthetic data can be an invaluable tool to help solve the problem. But how does it work?
Getting Started
Getting access to large volumes of historical market data for model building in the financial services industry can be cost-prohibitive. Similarly, using consumer financial transaction data for testing models in the retail banking sector requires the sharing of sensitive personal information with internal and external data analysts.
Generating synthetic data is a way around these challenges. The most popular models for generating synthetic data are generative adversarial networks (GANs), which have two sub-models: the generator and the discriminator, both of which work against each other to produce fake and real data.
Linked table synthetic data generation is particularly powerful for behavioral or time-series datasets, where different rows often have dependencies (e.g. pickup and drop-off times). To synthesize these types of datasets, the MOSTLY AI platform combines multiple models to produce a random distribution that is statistically similar to the original dataset. During this process, outliers are removed to avoid membership inference attacks. This is done automatically by the MOSTLY AI system.
Generating Test Data
Generating test data reliably takes time and requires specialized skills. A variety of tools exist for generating datasets and can be used to augment actual production data or serve as a substitute for sensitive real-world information for testing, analytics, development, and training purposes.
For some use cases, real data is too rare to collect (e.g., road accidents that self-driving vehicles must respond to) or dangerous to share (e.g., personal financial transaction data). In these cases, synthetic data can be used.
Unlike manually generated mock data or fake data, AI-generated synthetic test data is structurally representative, referential integer data with support for relational structures and can be easily generated on demand. This makes it more reliable than real-world data and ensures that you have enough variety to cover all possible test scenarios. It’s also an ideal substitute for real data that is too sensitive to share or that violates privacy regulations like HIPAA, GDPR, or CCPA.
Using Test Data
The process of creating synthetic data allows testers to address issues that may be difficult or impossible to resolve with real-world data. It also eliminates the need to spend time and money collecting, masking, or importing real-world data for testing purposes.
One example of this is data imputation, where a model replaces missing values in the dataset. This is often used to handle missing data from surveys or other sources.
Fully synthetic data is also ideal for hardware-in-the-loop testing (i.e., autonomous vehicle testing). This kind of test requires a high level of realism, which is incredibly expensive to recreate with real-world data.
For instance, to simulate road accidents for self-driving cars, a highly detailed virtual world must be created and sensor outputs must be simulated in a physically accurate way. This is extremely compute-intensive, even with a powerful cloud computing solution like GenRocket. This is why most hardware-in-the-loop testing is conducted with synthetic data.
Managing Test Data
Whether your teams are working on software testing, security testing or data analytics; they all need access to real and relevant test data. But often that’s difficult or impossible to get, especially for organizations that deal with personally and privacy sensitive information.
Manually creating the right kind of data for different test cases can be very time consuming and costly. This is why it is important to have tools available that make the process faster and easier.
Synthetic data generation can help reduce the need to store and reprocess real production data in your QA environment, as well as reduce infrastructure costs by using a subsetting model. And, it also makes it much easier for QA and DBA teams to self-refresh their test environments by offering them the ability to automatically refresh data with the push of a button. This is a great way to keep test environments consistent, accurate and up-to-date. This makes them a more reliable and efficient way to test and improve software.
Back in the old days, many teams of actuaries and underwriters did months of groundwork to unlock and analyse the information they had to calculate financial risks in order to issue an insurance policy.
Nowadays, I imagine, a lot of the grunt work is done by computers, allowing firms to hire fewer actuaries in order to compete in this crowded insurance space.
Maybe it's a tad bit easier to get hold of data on existing policy holders, but with GDPR and growing lack of trust in organisations, it is fair to say people aren't queuing up to share their personal data.
But my key takeaway from this discussion was that information I volunteered to give up was not only extremely private, but also very valuable (in the hands of profit-seeking insurance companies).
Secondly, any personal data collected - privately or publicly - will still need to be anonymised before it can be used.
Compliance becomes even more costly when you consider situations where data needs to be shared between different departments, team members working remotely in different locations, or even storing it in a cloud environment.
