Researchers from the University of Minnesota have devised a novel method of deploying Hollywood-esque motion capture technology to assist the 3D printing of sensors onto organs that expand and contract.
Printing directly onto moving soft tissues is challenging because sensors need to be able to adapt to the organ’s constantly changing parameters. The research team’s new technique on the other hand, overcomes this obstacle by using dual-cameras to create a toolpath for 3D printing in real time. Not only could this method be used by medical staff to monitor contagious patients from a safe distance, but according to the research team, it could even be applied to diagnose and monitor the lungs of those suffering with COVID-19.
“We are pushing the boundaries of 3D printing in ways we never even imagined years ago,” said Professor Michael McAlpine, lead researcher on the project. “In the future, 3D printing will not be just about printing, but instead be part of a larger autonomous robotic system. This could be important for diseases like COVID-19, where health care providers are at risk when treating patients.”
3D bioprinting and sensors
Recent advances in additive manufacturing have expanded the range of compatible 3D printing materials from conventional plastics and metals to include conductors and biomaterials. Despite this progression, AM has found it difficult to make an impact in bioprinting applications that target live biological surfaces, because they are typically soft and undergo constant motion and deformation. Moreover, because 3D printers rely on a prescribed design that’s manufactured offline and then transferred to the biological surface, this results in a mismatch, and the system is essentially fabricating “blinded.”
Manually adjusting the printing process is not a viable option either, as transfer processes can disrupt fragile 3D constructs such as hydrogel materials, and lead to costly human error. The researchers identified an alternative solution: in situ printing to seamlessly integrate the sensors on the target surface in an autonomous manner. In order to achieve this, a new closed-loop artificial intelligence (AI) was required, in order to adapt the fabrication process in real time by dynamically sensing the geometric states of the biological substrates.
The Minnesota research team had developed a closed loop system that tracked the motion of a non deforming hand in 2018, but this approach would require the development of a complex algorithm that could precisely track high dimensional deformation data. Instead, the researchers proposed that the space of deformation of the target surface could be “learned” from a dataset of 3D scans. This would allow accurate surface geometry to be recovered in 3D, via a set of fiducial markers tracked by a stereo camera system, and used to dynamically adapt the 3D printing toolpath in real time.
In order for this approach to succeed, the strain sensors used to calculate the deformation measurements needed to be compatible with the lung tissue surface, and the in situ 3D printing process. Conventional sensor designs are based on the dense packing of miniature sensor arrays, electrodes, and interconnects to improve resolution, but this approach is not compatible with the uncertainties that can occur during printing. Alternatively, the research team developed a new sensor using a combination of ionic hydrogels and Electrical Impedance Tomography (EIT) sensors, due to their high transparency and stretchability. To demonstrate their novel sensor and motion capturing technique, the researchers 3D printed a hydrogel-based EIT strain sensor directly onto a breathing lung to monitor its deformation.
The Minnesota team’s motion capture method
While the researchers could have used a stereo camera system to recover the varying 3D geometry of the organ’s surface in real time, its reconstruction algorithms would not be accurate enough, as submillimeter-level precision is required to avoid tissue injury. Alternatively, the team devised a two stage procedure whereby the system learned a parametric model of the surface geometry, which was supplemented by an estimate of the lung’s parameters using a set of fiducial markers.
In the first step, surface deformation was modeled using the movement of 12 fiducial markers, and 3968 waypoints which were extracted to create the printing toolpath. Before projection of the planar toolpath, corrections were made so it could reflect the physical growth in the size of the sensor when the surface expanded, and the shrinkage when the surface contracted. During the second phase, two machine vision cameras were mounted onto the 3D printer’s extrusion head for the real-time monitoring of the sensing process. The time series of parameters reflected lung perspiration, and were used to estimate the time-varying conformal toolpath that was adaptive to the tissue’s rigid-body motion.
After the adaptive toolpath was transferred to the 3D printer, the extrusion nozzle followed user-specified printing speed profiles and the resampled waypoints. The incorporation of deformation estimation reduced the mean error of shape modeling from 0.3 mm to below 0.02 mm, and the total error was within the ±0.8 mm tolerance. In addition, the ionic hydrogel ink displayed shear thinning behavior with decreasing viscosity above the shear rate of 0.1 s−1. This lower viscosity enabled the smooth extrusion of the ink from the printing nozzle under pneumatic pressure, and improved its controllability. After cross-linking using UV light, the hydrogel ink also demonstrated tissue-like stretchability according to uniaxial tensile test results.
The EIT sensor, consisting of a continuous thin layer of hydrogel and copper electrodes, was embedded in a soft silicone ring that could form chemical bonds with the hydrogel, to maintain stable hydrogel-electrode interfaces. During testing, forty measurements were taken from adjacent pairs of electrodes for one complete estimation of the strain map, and the maximum mean error recorded was 5.25%, similar to that of CT scan reconstruction. In order to demonstrate the 3D printing capability of their new in situ AI-powered system, the researchers fabricated the EIT strain sensor directly onto a porcine lung.
Experimentation and future applications
To simulate the organ’s deformations, the lung’s trachea was connected to a digital pneumatic regulator, and the surface geometry under each deformation state was then sampled by a structured light scanner. Using a custom-built 3D printing gantry system, electrodes embedded in a silicone ring were then attached to the printed layer, and exposed to UV light for cross-linking. Spatial mapping of the lungs was estimated and displayed in real time, which captured the cyclic contraction of the organ successfully. Not only were the sensors able to adhere to the lung surface under repetitive deformation, but they could be summarily removed without tissue damage or leaving residue.
As a result, the research team had successfully demonstrated the in situ monitoring of organ deformation, with a soft sensor that was 3D printed in vitro onto a breathing lung. The new method integrated offline machine learning with online computer vision-based tracking, and a conductive hydrogel ink with an EIT sensing configuration, to achieve this result. Although the team acknowledged that the biocompatibility of the sensors could be enhanced, the method’s precision improved, they suggest it could open new surgical applications for bioprinting. For instance, in clinical scenarios where injections of biological materials such as surgical glue and skin grafts are required, in situ autonomous 3D printing could replace manual operation, to achieve precise spatial control over longer durations.
Sensors in 3D bioprinting
3D printing has been used to create a range of different biosensors for medical monitoring purposes in recent years.
Researchers from Sungkyunkwan University for instance, used a commercially available inkjet printhead to fabricate wearable medical biosensors for personalized health monitoring in May 2020. Using a soft, flexible silicone elastomer with a sugar scaffold, the team were able to print high-resolution features in a lightweight, electrically conductive package.
Georgia Institute of Technology and Hanyang University scientists developed the first aerosol jet printed (AJP) biosensor for wireless monitoring of blood flow in August 2019. The research team 3D printed an implantable and stretchable electronic system, capable of monitoring blood flow in the brain for aneurysm treatments.
In December 2018, Washington State University (WSU) scientists leveraged 3D printing to create a more effective glucose screening device for diabetic people. Attached to the skin, the WSU biosensor was capable of monitoring glucose content in a person’s sweat rather than relying on their blood.
The researchers’ findings are detailed in their paper titled “3D printed deformable sensors” published in the Science Advances journal on June 17th 2019, and co-authored by Zhijie Zhu, Hyun Soo Park and Michael C. McAlpine.
You can now nominate for the 2020 3D Printing Industry Awards. Cast your vote to help decide this year’s winners.
To stay up to date with the latest 3D printing news, don’t forget to subscribe to the 3D Printing Industry newsletter or follow us on Twitter or liking our page on Facebook.
Looking for a job in the additive manufacturing industry? Visit 3D Printing Jobs for a selection of roles in the industry.
Featured image shows the researchers’ sensors being 3D printed directly onto a porcine lung. Image via University of Minnesota.
