Sphero Swarm

Goal & Overview

We are partnering with Imagination Station and Professor Severin to create an automated robotics-based educational display for students. Our goal is to use a swarm of ~30 robot "Sphero" balls to produce visually appealing and interactive molecular formations on a grid. We plan to also have a graphical user interface for users to interact with the display, where they will be able to choose which chemical formation to display, and by the simple click a of button, watch the autonomous movement. We hope this unique application will enhance education for children in a fun and visual mode!

The Sphero Bolt is a spherical robot that includes a gyroscopic sensor, accelerometer, an LED matrix, among other features. It can be connected and interfaced through the Sphero application, however, in order to control the large among of robots, we bypass the UI.

An example molecular formation we aim to create is shown below:

This project consists of five sub-teams: controls, algorithms, perception, hardware, and interface. Controls handles the connection and movement of the balls on the field. Algorithms works on path planning and mapping out the most optimal route for the robots to take when displaying each formation. Algorithms also takes into account error correction in the case of a faulty movement occurring. We plan to have a camera mounted over the display, which Perception (focused on computer vision) uses to track the Spheros during their movement, and send the relative positions back to algorithms. Hardware handles the design and prototype of the grid the robots move on, integration of the charging stations, as well as any other non-software related task. Lastly, Interface creates the user interface desiged to easily choose different molecular formations and interact with the display.

What have we been up to?

Controls

• Successfully connected to and able to control at least 6 robots simultaneously through multithreading

• Completed thorough testing on the Sphero balls

• Discovered and learned spherov2 module to control balls

• Integrated server-client system to provide an easier method for Algorithms to send instructions to the sphero

• Developed interface to provide ease of implementation for Algorithms to communicate to Controls

Algorithms

• Ideated position-based, pseudo-random algorithm to simulate polymer step growth

• Built a simulation of a test grid and test Spheros creating a molecular formation using Pygame

• Created the list of polymers we hope to display

Perception

• Optimized circle detection algorithm for identification of balls

• Created custom algorithm to identify and track 3+ robots on a field using OpenCV and Python

• Looked into camera options and set up for placement of camera on field

Hardware

• Conceptualized field and charging layout (charging for sphero bolts)

• Manufactured first full field prototype

• Created node and path design for field

Interface

• Used Flutter to create the interface that will be used to choose molecules to form

• Created multiple screens such as the Molecule List View and Molecultes View on the app

Plans for the Future

Controls

• Successfully connect and control 5+ Sphero bolts

• Implement a stream set up for commands; potentially rospy

• Begin feedback control to optimize movement

• Interface with Hardware about field material to slow ball movement

Algorithms

• Optimize and test position-based algorithm on 6 Spheros on grid prototype

• Implement more chemical accuracy in simulation; for example, limited certain "randomness" in initial randomized state to align to chemical properties

• Implement error-correction into algorithm with Perception feedback after each node-to-node movement

Perception

• Deal with edge cases in tracking algorithm; currently there are issues regarding having a set number of balls and balls leaving and returning the screen

• Interface with algorithms about error catching and the format of the positions to send back to them

• Improve precision in algorithm; essential since most of perception will be to optimize the node-to-node movement of each Sphero

• Test tracking algorithm on 5+ Spheros

Hardware

• Integrate charging station concept into grid prototype

• Integrate perception camera into display and design mount

• Complete final designs of each concept

Interface

• Finish designing all pages of user interface

• Finalize list of molecules (interface with algorithms)

• Work with Controls regarding sending bluetooth signal to Spheros