Fizyr, a computer vision company, and Imaging Development Systems GmbH (IDS) announced a solution to enable robots to pick unknown objects, even in harsh logistics environments. The result is an automated vision solution that enables logistic automation in various conditions and applications, like item picking, parcel handling, depalletizing, truck unloading or baggage handling. To complete the system with the optimal hardware, Fizyr integrated compact Ensenso 3D cameras in combination with high performance GigE uEye cameras from IDS.

According to IDS, more than 70% of labor in warehousing is dedicated to picking and packing, with companies gradually investing in logistics automation. Fizyr created a plug and-play modular software product that integrates smoothly with any system, giving integrators the freedom to choose the best hardware (e.g. Cobots or Industrial robots) for their picking cell. Depending on the individual customer application, up to four Ensenso 3D cameras in combination with powerful GigE uEye CMOS cameras can be used.

The uEye GigE CP camera takes 2D images of objects and provides them as input for Fizyr's algorithms, which then classifies the objects under the camera. The objects can be unknown and varying in shape, size, color, material and stacking. The Ensenso camera creates the point cloud maps and Fizyr's software combines it with the information from the 2D image, analyzing the surface of the cloud for suitable grasp poses for the gripper (or multiple grippers) and proposes the best ones. A clear representation of surfaces for different materials is critical, as it is a key component of its algorithms.

The specific camera models, as well as the number of cameras per system, depend on the individual use case of the customer. For a typical bin-picking solution with a cobot, one Ensenso N35 is used in combination with a GigE uEye CP, but there are clients that use one Ensenso X36 and a GigE uEye CP for bin picking together with four Ensenso N35 cameras for stowing the item in other bins.

The recommended uEye CP stands for "Compact Power" and is the tiny powerhouse for industrial applications of all kinds. It offers functionality with extensive pixel pre-processing and uses internal 120 MB image memory for multi-camera systems. The camera delivers data at full GigE speed and enables single-cable operation up to 100 meters via PoE ("Power over Ethernet").

The integrated IDS industrial cameras ensure a reliable, precise image capture needed for Fizyr's 1/5 software algorithms, which provide over 100 grasp poses each second, including the classification to handle objects differently. The software also performs quality controls and detects defects to prevent damaged items from being placed on a sorter — with the help of IDS cameras as sharp eyes of the automatic system.

These algorithms are able to provide all relevant information about segmentation and classification of the type of parcel, including box, bag, envelope/flat, tube, cylinder, deformable, etc. The system recognizes outliers or non-conveyables (i.e. damaged goods), best possible grasp poses in 6 DoF (degrees of freedom) and multiple ordered poses per object. It allows sensors or robots to deal with closely stacked or overlapping objects, highly reflective items and apparel in polybags, white-on-white and black-on-black flats, as well as transparent objects. The system segments unknown parcels, even under harsh lighting conditions.

"Fizyr normally uses one camera per system, usually uEye cameras in combination with Ensenso N35 and X36, but there are no limitations. The most common use case for Fizyr so far is one uEye and one Ensenso per system," says Herbert ten Have, CEO at Fizyr.

The different Ensenso models dispose of a light-intensive projector and produce a high-contrast texture on the object surface by using a pattern mask, even under difficult light conditions. The projected texture supplements the weak or non-existent object surface structure, called “Projected Texture Stereo Vision”. The result is a more detailed disparity map and a more complete and homogeneous depth information of the scene. Stereo Vision quality directly depends on the scene’s light condition and object surface textures and finding and calculating coordinates of corresponding points on less textured or reflecting surfaces is difficult.

Ensenso 3D camera models qualify for the wide range of demanding applications in the supply chain and logistics industry, such as 3D object recognition, classification and localization (e.g. quality assurance, commissioning), logistics automation (e.g. (de-)palletizing), robot applications (e.g. bin picking), capture of objects up to 8 m³, (e.g. pallets) or automatic storage systems.

"Fizyr has integrated the Ensenso SDK in its software using a modern and fast wrapper," says ten Have "A great advantage is the 2D/3D combination, which allows the image from the 2D camera to be placed over the 3D point cloud as an overlay".  A vivid impression of the scene and the camera image can be adjusted to the robot coordinate system by means of "hand-eye calibration" in order to ensure a target-oriented gripping.

The company says the strong growth of e-commerce has increased the demand for larger packages. At the same time, the constraints imposed by the global pandemic have led both to a reduction in the available labor force and to a need for physical distance in companies. To meet these challenges, companies are increasingly turning to robotic automation, in which artificial intelligence and machine vision play a crucial role. At the same time, this reduces costs and increases occupational safety.

The Fizyr-IDS solution offers the following:

• Enables easy integration into any system as a plug-and-play solution
• Hardware-agnostic and enables customers to adapt their order picking system to their needs at any time
• Precise, flexible and easy-to-use 3D vision system that provides optimal object information

For more information visit www.ids-imaging.us and www.fizyr.com.