Introduction
Cotton harvesting is a very challenging task. The theme of the ASABE Robotic Design Challenge is cotton harvesting in three-dimensional space. A variety of density, height, and positioning of cotton bolls are present in the competition, requiring the robot to not only correctly distinguish the cotton bolls from the simulated unripe cotton, but also accurately measure the position of the cotton to remove it from the husk.
Cotton harvesting is a very challenging task. The theme of the ASABE Robotic Design Challenge is cotton harvesting in three-dimensional space. A variety of density, height, and positioning of cotton bolls are present in the competition, requiring the robot to not only correctly distinguish the cotton bolls from the simulated unripe cotton, but also accurately measure the position of the cotton to remove it from the husk.
Vision
An Intel Realsense D435 stereo camera is used in the robot for the imaging and the machine vision detection. The camera outputs RGB and Depth data. The color image is fed to a machine learning model for cotton detection, and the corresponding depth is used to determine the 3D coordinates of the cotton
A 5-degree-of-freedom robotic arm is designed for the actuation to harvest the cotton. A custom spiked four "finger" gripper utilizing a slider-linkage mechanism, powered by a pair of micro linear actuators remove the cotton bolls from the husks 8400M
MobileNet SSD (Single Shot Detector) is trained for detecting the cotton bolls with the competition setup. The RGB color image from the realsense is sent to the SSD for detection where the SSD return bounding boxes for the detected cottons. The detector is configurated to only read the bounding box with high confidence The inference performance is at 13 FPS at the Nvidia Jetson, which is pretty fast for this competition
Nvidia Jetson Nano Developer kit (4GB) is used as the center processor for this robot. The Jetson nano host the Robotic Operating System (ROS) melodic which controls and forms the center pipeline and decision maker. THe Ketson also host the Machine learning Inference engine for the objection detection. An Arduino Mega and Arduino UNO are implemented to control the motors and the robotic arm and also interface with proximity sensors
Harvest
Detection
Processor
.png)