Vision System

1. Shallow Blue Lego Plays Chess Debra Burhans, Michael Lanighan, Robert Selkowitz and JerodSikorskyj Canisius College Buffalo, NY 14208 Vision System • The vision software of Shallow Blue is written in C++ and uses OpenCV (opencv.willowgarage.com/wiki/.) • First, a photo of the empty board is taken and all interior chessboard corners are located using the cvFindChessboardCorners function. Outside row and column corners are computed using linear approximation to the two closest interior coordinates. For each of the four corner coordinates of a chess square, a rectangle is maximally inscribed and stored for processing. The inscribed rectangle has a central region of interest (ROI) comprising approximately 1/16 of the inscribed rectangle’s area. This region does not suffer from overlap effects from tall pieces in adjacent squares. • Next, a photo is taken of the starting board configuration and the robot’s start color is determined by camera orientation and by averaging the color for the first two rows of each player. • Photos are taken for each move, where every previous and current square ROI has histograms computed and correlated using the cv::calcHist and cv::compareHist methods. Typically, correlations are < 5% for changed squares and > 75% for unchanged squares. • A list of changed squares is kept and analyzed for three cases: • 1) two squares have changed (move, capture, promotion) • three square have changed (en passant) • four squares have changed (castling) • else vision processing is marked as an error. In case 1), the square that becomes empty is determined by correlation with the empty board state. In cases 2) and 3) rule-based logic is used to determine details of the move.  For promotion, the human opponent is asked to input choice of piece. • Chess Engine • Houdini v 1.5a • One of top two chess engines in the world • UCI – Universal Chess Interface protocol • Only runs on Windows • Closed source • Nearly as strong as IBM’s Deep Blue • Software System • The robot loops over the following steps until end of game: • 1) Take photo of board before opponent moves. • 2) Opponent moves and presses laptop key. • 3) Take photo of board showing opponent move. • 4) The vision subsystem processes images and reports opponent’s move. • 5) If vision does not return a valid move after two attempts, the human operator is prompted to enter the opponent's move. • 6) Opponent’s move is sent from the vision subsystem to the chess reasoning subsystem. • 7) Chess reasoning determines the robot’s move and sends to the movement subsystem via Bluetooth • 8) The robot places and removes piece(s). If an error occurs, piece placement is hand-corrected. • Claw • The claw centers pieces as they are placed. It comprises one moving and one stationary finger. The moving claw finger clamps to the stationary one, holding the chess piece just right of the center of the square. As the piece is released, it is pushed slightly left, centering it in the square. Rubber pads on each finger help with grip, which is highly robust for all pieces except knights. The sloping shape of the knight and lack of a ridge ringing the upper portion of the piece make knights difficult to hold. • Movement • To move a chess piece, an electronic subsystem had to be coupled to the mechanical structure. Two Lego Mindstorm NXT bricks provide all the local control to the motors and color sensors. The first brick, SB1, is attached to the bridge topside on the left, from the robot’s point of view. SB1 connects to the left motor (Port A) and right motor (Port C), with colors sensors on the left (Port 1) and right (Port 2). It is important to note that each NXT motor contains an internal rotation sensor, capable of +/-1 degrees of accuracy for a single movement (under LeJOS regulated motor software). Also, each Lego Color Sensor has an approximate sensor speed of 2.5ms. The second brick, SB2, is mounted on the tower. Motor A controls claw manipulation, while Motor B raises and lowers the claw. Finally, Motor C moves the tower across the board columns. Like SB1, internal rotation sensors are included for each motor. Additionally, the color sensor on Port 1 is used to sense column position relative to the bridge. Hard-line RJ10 cables connect all sensors and motors to the bricks, including two extra-long custom cables built by the Shallow Blue team. • Tower • •All tower and claw movements are controlled by the SB2 NXT brick.  • •The tower serves two functions: translation in the horizontal direction, and claw control.  • •Movement of the tower along the bridge is handled by a similar stripe reading system to the one used for bridge motion along the rails.  In order to insure reliable motion, counterweights were added to one side of the tower.  This reduced oscillatory motion of the tower structure and minimizes the likelihood of a position misread. • •Claw height control is carried out by a motor fixed to the tower.  A pulley and string system lifts and lowers the claw through a guide shaft.   The depth to which the claw is lowered is determined by the piece to be carried.  Special emphasis was placed on insuring that any type of piece can be lifted over any other type of piece. • •Claw opening and closure is controlled by a motor which moves with the claw.  • References • Selkowitz, R. and Burhans, D. Shallow Blue: Lego-based Embodied AI as a Platform for Cross-curricular Project Based Learning , Proceedings of EAAI 2014. Quebec City, QC, July 2014. • D. Burhans, R. Selkowitz, J. Sikorskyj, and M. Lanighan, Shallow Blue: A low-cost platform for embodied chess play. Proceedings of the 6th Annual IEEE International Conference on Technologies for Practical Robot Applications (TePRA). Boston, Massachusetts, April 2014. • Lanighan, M., Sikorskyj, J., Burhans, D., Selkowitz, R. 2011. Lego Plays Chess: A Low-Cost, Low-Complexity Approach to Intelligent Robotics. Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence. San Francisco, CA. AAAI Press. • Matuszek, C., Mayton, B., Aimi, R., Deisenroth, M.P., Bo, L., Chu, R., Kung, M., LeGrand, L., Smith, J.R., Fox, D. 2011. Gambit: A Robust Chess-Playing Robotic System. Proceedings of ICRA 2011. Shanghai, China: IEEE. • Sokic, E. and Ahic-Djokic, M. 2008. Simple Computer Vision System for Chess Playing Robot Manipulator as a Project-based Learning Example. In Signal Processing and Information Technology, 2008. ISSPIT 2008, 75-79. Sarajevo, Bosnia and Herzegovina: IEEE.