Why Build a Testbed?
Technology testbeds reduce the chance of failure during full-scale deployment by providing a controlled environment to identify risks and run experiments without disrupting production. Generally, testbeds use scaled-down equipment to minimize costs while still mirroring key functions and capabilities of full-sized systems.
Pilot Plants
All testbeds prove out new manufacturing processes, but pilot plants do so on the largest scale. Flagship facilities are typically new constructions, employ at least some full-time staff, and require significant capital investment. They are often built in partnership with government or economic development organizations. At a pilot plant, iterative design, testing, and experimentation from researchers, scientists, and engineers leads to close-to-correct process parameters that will carry over to full-scale production.
Examples of Pilot Plants
In 2024, the Supercritical Transformational Electric Power (STEP) test facility opened at Southwest Research Institute in San Antonio, Texas, with support from the National Energy Technology Lab (NETL). By demonstrating supercritical carbon dioxide (sCO2) turbine technology, the pilot plant validated the efficiency, scalability, and environmental benefits of the sCO2 Brayton power cycle. Operating at 4 megawatts of grid-synchronized power, it provided insights into turbine performance, heat management, and system integration. The demo facility enables real-world testing of equipment, process designs, and operational conditions.
From 2019 to 2022, VTT Technical Research Centre of Finland Ltd. built a fiber processing pilot plant to develop a semiautomated process for generating digital twins. The plant used detailed simulation and a controlled environment to uncover and address real-world challenges like integrating legacy systems and automating data extraction. This approach minimized risks and costs while accelerating digital transformation, demonstrating how pilot-scale setups are instrumental in bridging theoretical concepts to industrial-scale implementations.
Other pilot manufacturing plants announced or opened in 2024:
Retrofit Testbeds
Unlike a pilot plant, these won’t make headlines for new job creation or regional economic gains. Testbeds at this scale are common at universities and research institutes. Industry consortiums can also be an avenue to collaborate on late-stage R&D or early commercialization of promising new technology.
Examples of Retrofit Testbeds
Starting in 2016, the National Institute for Standards and Technology (NIST) hosted the Smart Manufacturing Systems testbed to facilitate research on product life-cycle data. The testbed pulls data from 12 machine tools located at the NIST campus machine shop in Gaithersburg, Maryland. Equipment connected to the testbed includes vertical and horizontal machining centers, mill-turn machines, and a CMM. The machine shop supplies parts to NIST researchers, and the connectivity and software components of the testbed are layered on top of the existing equipment, scheduling, and process management software tools.
In 2022, a digital twin framework to enhance manufacturing process management was deployed at MxD in Chicago. The framework standardized data collection and analysis to improve visibility and control of production operations. The project also tested a "mobile worker" solution that enabled equipment operators to receive tasks, access critical data, and update status in real-time on mobile devices. The testbed allowed manufacturers to evaluate AI-driven analytics, predictive maintenance, cybersecurity, and more in a safe, low-risk environment, helping them understand each tool before committing to full-scale adoption. By mirroring real-world processes, the testbed supported training and skill development, allowing workers to build familiarity with new tools and techniques.
Benchtop Testbeds
The same principles behind pilot plants or testbed retrofits can also be applied to self-contained desktop- or benchtop-scale testbeds. When they are developed by a company, these small-scale tools often go unpublished. For those that are built or codeveloped in a university setting, writeups are found in academic literature.
Examples of Benchtop Testbeds
Limitations of Technology Testbeds
Because they are intended to mirror real-world manufacturing conditions, building and maintaining a testbed comes with similar challenges. These include safety, data security and privacy, system integration, and physical and virtual infrastructure management. Resource constraints like limited space, funding, and stakeholder engagement can also slow progress.
In addition, good experimental design is also required. Test cases need to be clearly specified and well documented. Results need to be well-articulated and easy to circulate. Otherwise, there is a real risk that findings from a testbed are never scaled up or applied to production.
AMT’s Testbed
AMT – The Association For Manufacturing Technology built a “pop-up shop” with a desktop 5-axis milling machine in 2018 as a demonstration and research platform for machine connectivity and digital thread. Later, a series of collaborative robots were added, and the original milling machine was upgraded to a newer model. In addition to staff training, active research areas on the AMT testbed include:
Specifications and capabilities modeling in MTConnect
Remote access, safety, and security for industrial IoT
Standards harmonization (ROS, OPC UA, MTConnect, MQTT)
Equipment list:
Penta Machine Co. PocketNC v2 desktop 5-axis CNC mill (with enclosure)
IGUS ReBeL6 robotic arm + Schunk gripper
Intel RealSense D435i depth-sensing stereo camera
Sonicwall firewall
Dedicated Windows workstation for CAD & simulation, cell control, and remote access
The AMT testbed is set up for robot-tended machining. It also has a data collection system based on MTConnect and secure, account-based remote access to the cell.
This article was written by Chloe Radigan in collaboration with Russ Waddell, with additional support from Benjamin Moses and Stephen LaMarca.
Publications:
“Low cost development testbeds for implementing the digital thread” https://nvlpubs.nist.gov/nistpubs/ams/NIST.AMS.100-24.pdf (Pages140-144)
“Authorship approach and production method for machine-readable manufacturing data standards” https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=929601
“The affordably connected factory: A brief evaluation of sensors and hardware deployed in industrial applications” https://www.sciencedirect.com/science/article/pii/S2351978921001074
“Industry review of distributed production in discrete manufacturing” https://asmedigitalcollection.asme.org/manufacturingscience/article-abstract/142/11/110802/1082723/Industry-Review-of-Distributed-Production-in
“Capability modeling for smart manufacturing standards development” https://nvlpubs.nist.gov/nistpubs/ams/NIST.AMS.100-29.pdf (Pages191-194)
