Robotics, Autonomous Systems, and Control - CSM Capabilities

Robotics, Autonomous Systems, and Control - CSM Capabilities Kevin L. MooreG.A. Dobelman Distinguished Chair and Professor of Engineering Director, Center for Automation, Robotics, and Distributed IntelligenceDivision of EngineeringColorado School of MinesGolden, Colorado

Outline • Introduction • Robotics • Other Control-Related Research • Sensor Networks • Renewable Energy-Related Control Research

CARDI • 14 faculty, interdisciplinary • Electrical, Mechanical, Civil • Computer Science, Math • Bio-medicine, Bio-mechanics Colorado School of Mines Located in Golden, Colorado, USA 10 miles West of Denver CSM sits in the foothills of the Rocky Mountains • CSM has about 300 faculty and 4000 students • CSM is a public research institution devoted to engineering and applied science, especially: • Discovery and recovery of resources • Conversion of resources to materials and energy • Utilization in advanced processes and products • Economic and social systems necessary to ensure • prudent and provident use of resources in a • sustainable global society

Degrees at CSM (Grad and Undergrad) • Chemistry • Chemical Eng • Economics/Business • Engineering • Environmental Science/Eng • Geochemistry • Geology and Geological Eng • Geophysics and Geophysical Eng • Math/Computer Science • Material Science • Mining and Earth Systems Eng • Metallurgical and Materials Eng • Petroleum Engineering • Physics

CSM Engineering Division • B.S. in Engineering with Specialties in • Civil • Electrical • Environmental • Mechanical • M.S/Ph.D. in Engineering Systems • Civil • Electrical • Environmental • Mechanical • Bioengineering and Life Sciences Minor • Includes biomedical engineering, biophysics • Pre-Medical & Life Sciences • 5-year Programs (BS+MS) • Engineering Physics • Engineering Systems • Environmental Science • Humanitarian Engineering Minor • The application of math, science, and engineering to improve the wellbeing of under-served populations

CSM Research (Control-Oriented view) • Materials • Nuclear • Welding • Energy • “Traditional” (e.g., Petroleum Institute) • Combustion • Renewable • Colorado Fuel Cell Center • Power Electronics for Hybrid Renewable System • Solar (PV materials, thermal, systems) • Wind • Center for Space Resources • Insitu resource utilization • Lunar, Martian exploration • Geo-Sciences/Mining • Environmental Sciences/Hydrology • “Intelligent” Geosystems • EE and CARDI Activities

Power Systems Signal and Image Processing Renewable Energy Power Electronics Robotics Control Systems Humanitarian Engineering EE Research Areas and Applications Wireless Communications

EE Research Areas and CSM’s Mission • Robotic welding • Welding process control • Control of plasma processes • PV manufacturing process control • EARTH’S REOURCES • Discovery and • recovery • MATERIALS • Utilization in advanced • processes and products • Robotics for mining • - Image processing • Mine safety • - Control systems • - Sensor networks • ENERGY • Generation, conversion, and distribution • Power transmission and distribution • Renewable energy • - Wind power • - Fuel cell control • - Hybrid power system coordination • ENVIRONMENT • Economic and social systems necessary to ensure prudent and provident use of resources in a sustainable global society • Humanitarian engineering • Robotics for security applications • Biomedical engineering • - Image processing

Center for Robotics, Automation, and Distributed Intelligence (CARDI) • CSM research center focused on • Control systems, robotics, sensing (especially vision) and communication networks, machine learning and intelligence, ad hoc mobile networks, sensor networks • Applications to problems of concern at CSM, including: • Environment, energy, natural resources, materials, transportation, structures, geotechnical, information, communications, networking, medicine, and data mining • Problems requiring multi-disciplinary systems approach to integrate technologies from the different disciplines • 14 faculty • Electrical, mechanical, civil, computer science, mathematics

Biomedical Environment 3D Stereo Mapping Medical Imaging Sensor Networks Control Stereo-Vision for Welding Automation Wind Turbine Control Reconfigurable Wireless Nodes Activity Identification Data Mining Automation Distributed Intelligence CARDI Research Areas and Applications Robotics Mobile Ad-hoc Networks Communications Control of Plasma Processes Manufacturing

CSM Control Systems-Related Research • CSM EE/ME/CE/CARDI faculty conduct both theoretical and applied research in control systems: • Fault detection and identification (Vincent) • Control of material processing (Moore, Vincent) • Autonomous systems and robot control (mobile, arms, and mobile arms) (Moore, Steele, Vincent, Hoff) • Coordinated control for robotics and UAVs (Moore) • Vision-based navigation and control (Moore, Vincent, Hoff) • Augmented reality (Hoff, Vincent) • Intelligent sensing for geo-systems applications (Mooney) • Sensor networks (Moore, Weiss, Colagrosso) • Welding control (Hoff, Moore, Steele, Vincent) • Learning control, intelligent control (Moore, Simoes) • Adaptive control (Johnson) • Wind energy applications (Johnson) • Distributed coordination/control of renewable energy (Johnson, Simoes) • Power electronics for hybrid energy control (Simoes) • Fuel cell system controllers (Moore, Simoes) • System engineering (Moore, Steele)

CSM Robotics • CSM has a long legacy of robotics-related activity • CSM faculty and staff have significant robotics-related expertise • Mechanism Design • Robotic Manipulators • Mobile Robots • Cooperative robotics • UAVs • Vision-based robotics; Augmented reality • User Interfaces • System engineering • Applications: mining and lunar exploration • Commercialization

Martin Marietta BatMobile NASA Contest SAE Walking Machine NSF REU Arm CSM Robots Concept design for In-situ resource utilization

Weederbot III 2005 Senior Design 2006 (supervised by John Steele) Autonomous Mower Project

Based on the ODIS Platform used in theatre Mobile Manipulators (Mobile ARMS) • Human-robot interaction • Autonomous Pick-and-Place • Visual Servoing

3D Models for Navigation and Manipulation Robotics can be used to automate load/haul/dump vehicles Automation of Load/Haul/Dump • Remove miners from operating area Stereo vision is used for • Guidance • Map building • Collision avoidance 50m Long Mine Model (green from laser ranger, yellow from stereo) Faculty: Steele and Vincent

Autonomous Robots built by Kevin Moore’s Team at Utah State T1 -1998 ODIS I -2000 T2 -1998 T4 -2003 T3 -1999 (Hydraulic drive/steer)

Autonomous Tractors and Unique Mobility Robots built by Kevin Moore’s Team at Utah State Automated Tractor Projects(CSOIS Spin-Off, Autonomous Solutions, Inc.) Unique Mobility Robots

“Putting Robots in Harm’s Way So People Aren’t” ODIS – the Omni-Directional Inspection System An ODV Application: Physical Security

From Intelligent Behavior to Cooperative Autonomy… • Seek a single machine that can do both of the following tasks via semantic (verbal) instruction from a (human) supervisor: Load a trailer Cooperatively weld a pipe • Humans can do this! • How do we make a group of robots that can also do this?

Mote-Based Distributed Robots Prototype plume-tracking testbed developed by Moore at USU Robotic wireless networks for remote video streaming - Weiss

Wireless 802.11g Video Streaming through a Robot MANET Laptop and Starting Position Robot Monitoring Software on Laptop (video feed from lead robot on upper right) Stopping positions of robots (blue dots) Lead Robot with Video Camera Faculty: Weiss

Other Control-Related Research • Faculty conduct both theoretical and applied research in areas related to robotics and control systems: • Welding Control (Moore, Vincent, Steele) • Material Processing (Moore, Vincent) • Intelligent Sensing and Control (Mooney, Steele) • Computer Vision (Hoff, Vincent) • Wireless system design (Weiss) • Sensor networks (Colagrosso, Moore, Weiss)

Robotic Welding Rectified Stereo Image Pair Faculty: Steele, Vincent, and Hoff Derived surface of weld pool

Welding Control and Control of Material Processing • Control of Materials Processing • Foundry Cupola • Gas Metal Arc Welding • PV Material Manufacturing

Smart Bit Technology: Force Sensing for Manipulation (Steele) Sensory Goals • In-Situ measurements • Bit wear indicator • Horizon detection • Performance indicators • Improve Mind-Machine link (20-mile Longwall) • Equipment Application • Radial borers • Continuous miners • Longwall shearers (Continuous Miner)

Mike Mooney Intelligent Soil Compaction (funded by NSF & NCHRP) Forward accelerometer eccentric V = 2-4 m/s 200-300 mm soil Layers of previously compacted soil; possible near surface bedrock Objectives:(1) Develop the relationships between vibration properties of roller compactor and underlying soil properties relevant to design (2) Improve our understanding of how feedback control might improve soil compaction (3) Improving parameter estimation via geolocation data, multiple passes, etc.

Current Trial's Input Current Trial's Output Plant Next Trial's Input Iterative Learning Controller Desired Output Iterative Learning Control (Moore) • Paradigm for systems that operate repetitively

Iterative Learning Control Applications • Iterative Learning Control • Paradigm for controlling systems that repeat the same operation over and over • New applications • Develop new approach called multi-pass ILC • Vision-based ILC (also for multi-pass problems) • nD ILC (e.g., irrigation control)

Vision/User Interface; Augmented Reality(Hoff) A system to recognize gestures, for the purpose of robot control Constructing scene models from stereo vision Augmented reality system developed at CSM Registration of range data

Activity Identification and Visualization (Hoff) Problem: Detect and identify unusual or suspicious activities in surveillance data Cars and bicycles are labeled green and yellow Statistical measures (eg, chi-square) signal how unusual a track is from the rest of the scene, and compare two scenes Interactive visualization tools enable analyst to pick salient features for machine learning Person handing out flyers is labeled red Faculty: Hoff and Lee

Research in telecommunications focuses on mobile computing and networking • Research areas: • How to connect mobile computers (laptops, palmtops, etc) to the Internet • How to create ad hoc networks on demand • one unit may be connected to some wired network • how to create and maintain network connectivity • previous infrastructure is demolished or non-existent • Quality of service Faculty: Camp, Navidi, Colagrosso, Liu

Potential Applications of Ad Hoc Networks

Legend: Flows of people, material Flow of information Flows of air Fixed radio node Toxic gas Mobile radio node Autonomously Reconfigurable Systems • Consider an underground mine

Legend: Flows of people, material Flow of information Flows of air Fixed radio node Toxic gas Mobile radio node Add sensors and flows of people, material, and air

Legend: Flows of people, material Flow of information Flows of air Fixed radio node Toxic gas Mobile radio node Suppose a hazardous gas develops

Legend: Flows of people, material Flow of information Flows of air Fixed radio node Toxic gas Mobile radio node The system should autonomously-reconfigure to redirect flows of people, material, and air

Sensor node VDSL node Phone cable See live demo at http://ore.mines.edu/~mcolagro/edgarmine/

Control-Related Energy Research at CSM • The Electrical Engineering group at CSM is involved in three main areas of electrical energy research • Distribution and transmission • Control and coordination • Power electronics • Other faculty in the Engineering Division and other CSM departments are also involved in electrical energy generation research.

The future of electrical energy • Energy will be introduced to the grid from a variety of sources • Distributed generation with non-traditional resources is less predictable and more difficult to control • Could eventually see emergence of the “smart grid” or “Enernet” (energy network) Figure courtesy of Ben Kroposki (National Renewable Energy Laboratory), and Janet Ginsburg (“Reinventing the Power Grid” in Business Week, February 26, 2001 pp.106-107).

CSM Electrical Faculty Energy Research Interests • A key factor for clean, efficient energy is management and control • Specific CSM EE area of interests include: • Control of distributed and conventional generation sources and associated manufacturing processes • Turbine control in wind; balance-of-plant control in fuel cells; combined thermal/power management systems in concentrating solar power; PV manufacturing • Control of power flow in hybrid energy systems • Coordination of turbines in a wind farm; “smart” inverters/converters • Control of power flow into and out of the grid • Anti-islanding/ prevention of cascading failures/“robust grid”; power quality/reliability; grid synchronization • Economically-driven grid/resource management • “Smart grid”; “Ener-net” = Energy + Network

Photovoltaic Solar Collector Field The “Grid” PV Field Sensor Network and Controller Switching/Inverter Substation & Power Conditioning Thermal Solar Energy Collector Field Auxiliary Storage Power Plant (Steam or Direct-to-H2) Integrated Roof w/ Thermal/PV Solar Energy Load Balancing Fuel Cells SCADA System Smart Data Bus Wind and Other Non-Solar Renewable Energy Generation . . . . . . Control Room

Photovoltaic Solar Collector Field Fuel Cells Wind and Other Non-Solar Renewable Energy Generation . . . . . . • PV Manufacturing • Tyrone Vincent • Bob Kee • Fuel Cell Control • Marcelo Simoes • Kevin Moore • Wind Turbine/Wind Farm Control • Katie Johnson • Marcelo Simoes • Kevin Moore

Substation-to-Grid Transmission • P.K. Sen • Sid Suryanarayanan • Auxiliary Units-to-Substation Transmission • P.K. Sen • Sid Suryanarayanan

Switching/Inverter Substation & Power Conditioning . . . . . . • Microgrids • Marcelo Simoes • Sid Suryanarayanan • Inverter Power Electronics • Marcelo Simoes Auxiliary Storage • Auxiliary Storage/Load Balancing/Coordination • Marcelo Simoes • Katie Johnson • P.K. Sen • Kevin Moore Load Balancing

Robotics, Autonomous Systems, and Control - CSM Capabilities