Scouting

Motivation

Teams' rankings and Offensive Power Ratings (OPR) are often not viable metrics for evaluating quality of performance at competitions. Additionally, they provide no detailed information about robots, such as their methods of scoring. Our scouting system provides higher accuracy by tracking the actions of every robot in every match, which allow us to make well-informed decisions. Additionally, our team has historically (pre-2016) not given enough attention or analysis to the data we collected. Now, our data is at the forefront of our match strategization and alliance selection.

Click here for an explanation of how OPR is calculated and here for an article about the benefits and limitations of OPR.

Scouting Procedure

Before competitions begin, pit scouters ask teams about their robot's capabilities. Pit scouters are chosen before competitions to ensure that they enjoy talking with teams and that they collect all the necessary information. During competitions, our team separately scouts for quantitative and qualitative information. Each match, six quantitative scouts and six qualitative scouts each observe an individual robot. Since 2016, our scouts have recorded their observations on paper scouting sheets, but this year we will be using an internally-developed app for the Kindle. Quantitative data is then given to the data analyst(s) in the form of an excel spreadsheet or .csv file. The data analyst(s) use Tableau, a versatile data visualization tool, to chart information quickly and effectively.

Often with paper scouting, a separate team member specifically entered data into Excel to be given to the data analyst(s). We designed our paper scouting sheets to be user-friendly for both the scouts and data entry person. We accomplished this by making the quantitative scouting sheet answers all numerical and asking scouts to copy their responses to a vertical list at the side of the sheets. The scouting sheets were also organized chronologically and included diagrams of the field. We will be using Bluetooth (as part of the internally developed app) to transfer data from the Kindles directly to a .csv file this year. An example scouting sheet is shown below.

With Tableau, the data analyst(s) and match strategist(s) prepares the drive team with knowledge about the strengths and weaknesses of our own alliance and our opposing alliances. We also use the data we collect to see which teams would make for good alliance members. An example chart for the 2018 game FIRST Power UP is shown below with anonymized team numbers. Teams that could not consistently score on the scale (e.g. team 35 in the chart) may opt to pick a team that focuses on the scale to complement their strengths, like team 22 or 24.

Influence

In the spirit of gracious professionalism, we provide our Tableau charts to the public at every competition we go to so that teams without their own scouting systems may be more informed about teams' performances. Our scouting system is recognized by most all of the Chesapeake District and adapted by a number of other teams.

Our scouting system has led us to great success in competitions as well. In the 2017 Chesapeake District Championships event, our team was ranked 52nd out of 58 teams, and yet was chosen 12th in the elimination rounds because we showed top-ranking teams about our real capabilities and made it to the 3rd round of grand finals, where one alliance member disconnected and our robot got stuck on a gear. As an alliance captain at girlPOWER in 2018, we chose the 13th and 14th ranked teams (out of 14 teams) due to the results we saw in our data, and went on to win the event.