An emerging artificial intelligence technique called “transfer learning” could help drug makers use data to speed up the development of biopharmaceutical manufacturing processes, according to new analysis.
In transfer learning, predictive models that have been trained on historical data are used to improve the performance of a task.
Unlike machine learning (ML)—where the training process begins from scratch—transfer learning applies existing knowledge to new but related problems, reducing the amount of data and time required to build the model.
Researchers at the Karlsruhe Institute of Technology in Germany, who looked at the approach, identified several potential biopharma applications, according to lead author Daniel Barón Díaz, citing reactor modeling as an example.
“Transfer learning models can be used to predict critical outcomes like viable cell density (VCD) and product titre from online sensor data—for example, pH, temperature, gas flow—from historical data from a different, but related process.”
The approach can also optimize process monitoring. Díaz tells GEN that, “Transfer learning-enhanced soft sensors can be established to monitor protein concentrations in real-time by leveraging existing models from related fermentations.”
Data limitation
When compared with other model-building techniques, transfer learning offers potential cost and time savings, according to Díaz, who cites a reduced experimentation burden as an example.
“Conventional machine learning requires large, structured datasets that are often unavailable in biopharma due to the high cost and labor-intensive nature of experiments. Transfer learning allows companies to leverage historical data and existing models to build reliable predictors for new processes with very limited data.
“By reusing prior knowledge, transfer learning can significantly decrease the number of experiments required—sometimes needing only one to three batches to achieve robust simulations,” he says.
However, the ultimate benefit is that transfer learning speeds up process model development, according to Díaz, who adds, “It can make model adaptation faster than retraining from scratch, facilitating quicker process design and digital twin deployment.”
Challenges
So, transfer learning has the potential to create predictive models for manufacturing development. However, the key caveat is that the processes involved must be sufficiently similar for it to be effective, Díaz says.
“For transfer learning to be effective, the source and target domains must be meaningfully related. If the processes are too different, the assumptions and learned representations may not align, leading to negative transfer, where the transferred knowledge actually degrades the model’s performance.
“Data sets obtained at different scales or under varying conditions are often inconsistent, which can hinder the successful transfer of knowledge. Fine-tuning complex neural network architectures on very small target datasets can lead to overfitting, where the model fails to generalize to new data,” he says.
To address this, manufacturers will need to establish metrics to determine similarity, Díaz explains.
“There are currently no standardized metrics for measuring domain similarity in bioprocessing, nor are there comprehensive benchmark datasets to easily compare different transfer learning techniques.”
Another challenge is the current lack of AI expertise in the industry, Díaz says.
“There is often a disciplinary knowledge gap between process engineers and data scientists, and ML models without a mechanistic backbone may be perceived as opaque black boxes, hindering trust and industrial adoption,” he tells GEN.
