The Deep Picking project, funded by the German Federal Ministry of Education and Research under grant number 01IS20005C, ran from 1 April 2020 to 30 September 2022, with a six‑month extension caused by pandemic‑related delays. The consortium comprised Fraunhofer Institute for Production Technology and Automation (IPA), Premium Robotics GmbH, drag & bot GmbH, and the automotive group Daimler Trucks AG as an associate partner. The project’s core objective was to develop a fully software‑driven, AI‑based picking system that can autonomously depalletise pallets and perform kitting—i.e., the separation of items from bulk containers—without manual re‑configuration for each new object type. The ambition was to create generic algorithms that remain applicable to a wide range of objects and application contexts, thereby enhancing scalability and economic viability in logistics environments.

Technically, the project delivered a suite of advanced perception, planning, and control modules. First, a six‑degree‑of‑freedom (6‑DOF) object pose estimation was implemented using folded neural networks. This estimator can localise diverse containers on colour‑pure pallets with tight tolerances, enabling precise pick‑and‑place operations with a roll‑and‑grab gripper from Premium Robotics. A second estimator was designed for bulk items stored in boxes, allowing self‑configuration for unknown objects. The model‑free pose estimation relied on supervised machine‑learning techniques, enabling the system to adapt to new shapes without explicit CAD models.

Once the pose was known, an automatic grasp pose generator produced feasible contact points for both suction and parallel‑jaw grippers. The system can autonomously decide which gripper type is most suitable for a given object, improving success rates. For model‑free grasping and item separation, dedicated search algorithms were developed, complemented by collision‑detection routines that ensure safe manipulation in cluttered environments. Validation procedures assessed graspability per object instance, grip stability, pallet stability, and overall success rates, including error handling.

Trajectory optimisation for robot arms was another key contribution, reducing cycle times and enabling efficient separation of jammed items within containers. An ordered placement algorithm was created to deposit model‑free grasped items onto a target surface. This algorithm re‑captures the item with a 3‑D sensor, re‑localises it in the gripper, and uses edge‑and‑plane search strategies to place it gently on the desired surface.

The project also established an online learning infrastructure. An online learning server and training pipelines were built to continuously refine the perception and grasping models using data from both simulation and real‑world experiments. This closed‑loop learning approach ensures that the system improves over time without manual intervention.

From a collaboration perspective, the Fraunhofer IPA led the research and development of the core AI algorithms, while Premium Robotics supplied the roll‑and‑grab hardware and contributed to the safety concept for the demonstrators. drag & bot GmbH focused on integrating the software stack and providing the robotic platforms for testing. Daimler Trucks AG, as an associate partner, supplied real‑world use cases, defined standard tests, and evaluated the system’s performance in a commercial setting. The project was organised into seven work packages with four milestones, allowing parallel progress on depalletising and kitting modules. The safety concept, interface specifications, and standard test protocols were all defined early, ensuring that the demonstrators met industrial safety and interoperability requirements.

In summary, Deep Picking produced a robust, AI‑driven picking framework capable of handling unknown objects in both palletised and bulk contexts, with demonstrators that validated the approach under realistic conditions. The collaboration among research, industry, and automotive partners, supported by BMBF funding, positioned the technology for future deployment and further development in logistics automation.