Part 1 Environment 1. Matlab 64-bits 2014b or later version(64-bits one is required) 2. Labview 2012 32-bit 3. Kinect Camera for Windows 4. Fanuc Robot (by NI)
note: open "RoboticBinPicking" file, add the "AddToPath" file to matlab path.
Part2 Grab the Calibration Marker
purpose: operate robot to grab your calibration marker Steps: 1. run labview.exe, open the file "MatlabLabVIEWFanucComm.vi" (RoboticBinPicking/AddToPath/MatlabLabVIEWFanucComm), and click "run" note: keep it running all the time 2. run matlab program, open file "LabVIEWMatlabComm" change "values_to_send = [350 00 0 -90 0 -90 0]; " to "values_to_send = [350 00 0 -90 0 -90 1]; " and run this .m program in order to open gripper 3. put your marker between the gripper set "change "values_to_send = [350 00 0 -90 0 -90 0]; " and run this program in order to grab the calibration marker. Part 3 Calibration purpose: to get the transformation matrix which describes a one-to-one mapping from camera coordinate system to robotic coordinate system. Steps: 1.run matlab.exe, open file "CalibrationPlusResampling.m" and run it note: watch you command window in matlab while run this program, because it will give you some instructions. 2. matlab will inform you to choose template. Now just move your mouse right upon the intersection points in the picture, click it, and press "enter" on your keyboard. note: this step is used to choose template. 3. while sampling, the command window will ask for your input. Just follow the instructions in command window. 4. After sampling 64 points, the program is still running. It will show the sampled center. You just need to compare if both the sampled center(marked red) and the real intersection point are overlapped. If overlapped, you can go to the next sampled point, however if failed, you need you input "3" (instructions will be there) to change to sampled center. 5. After these steps, the program stops. Now you have lots of variables in your working space. You only need the one called "TransMatrix". You can delete all the variables except "TransMatrix" and save this information.
note: Every time you move the camera, you need to do the calibration again.
Part 4 Picking purpose: operate robot to implement a pick up. Steps: 1. Load the "TransMatrix" information into your working space 2. open "RoboticMultiPicking" and run it 3. Put the markers (target waits to be picked up in the working space) note: the work space is not big enough, it is a certain areas according the moving range of the robot. It's better if you do not set the markers over a wide range. 4. Put the Bin to the place where you can see half of a circle on the white plane. 5. Follow the instruction in your matlab command window. (1) choose working space in the RGB image: move your mouse right upon the “upper left corner” of your working space, click it, and press "enter" move your mouse right upon the “bottom right corner” of your working space, click it, and press "enter" (2) choose the depth range look at the depth image, and enter the lower bound(just smaller than the depth value of the the most front marker) ; then enter the upper bound (just greater than the depth value of the farthest( to camera) marker).
(3) then begin picking. In this process, you can add some new markers.
Note: If you have difficulty understanding these steps or fail to implement a pick up, just feel free to send me an email.