Team rankings at competitions and Offensive Power Rating (OPR) are oftentimes not viable metrics for evaluating the quality of teams' performance at competitions. Additionally, they provide no detailed information about robots, such as their methods of scoring. Our scouting system makes up for this by tracking the actions of every robot in every match, which allow us to make well-informed decisions. Click here for an explanation of how OPR is calculated and here for an article about the benefits and limitations of OPR. Additionally, our team has historically (pre-2016) not given enough attention or analysis to the data we collected. Now, our data is streamlined for Tableau, a versatile data visualization tool we use to chart information quickly and effectively.
Before competitions begin, pit scouters ask teams about their robot's capabilities. Pit scouters are chosen before competitions to ensure that they enjoy talking with teams and that they collect all the necessary information. During competitions, our team separately scouts for quantitative and qualitative information. Each match, six quantitative scouters each observe an individual robot and two qualitative scouters observe the three robots on one alliance. Since 2016, our scouters have recorded their observations on paper scouting sheets, but this year we will be using an internally-developed app for the Kindle. Quanitative data is then given to the data analyst(s) to create charts in Tableau. An example chart is shown below.
Often with paper scouting, a separate team member specifically entered data into excel to be given to the data analyst(s). We designed our paper scouting sheets to be user-friendly for both the scouters and data entry person. We accomplished this by making the quantative scouting sheet answers all numerical and asking scouters to copy their responses to a vertical list at the side of the sheets. The scouting sheets were also organized chronologically and included diagrams of the field. We will be using bluetooth (as part of the internally developed app) to transfer data from the Kindles directly to a .csv file on the data analyst's computer this year. An example scouting sheet is shown below.
With Tableau, the data analyst(s) and match strategist(s) prepares the drive team with knowledge about the strengths and weaknesses of our own alliance and our opposing alliances. We also use the data we collect to see which teams would make for good alliance members. For example in the 2018 event, teams that were consistent at scoring on the scale may opt to pick a team that was better at switch and vault to complement their strengths. An example chart is shown below with anonymized team numbers.
In the spirit of gracious professionalism, we provide our Tableau charts to the public at every competition we go to so that teams without their own scouting systems may be more informed about teams' performances. Our scouting system is recognized by most all of the Chesapeake District and adapted by a number of other teams.
Our scouting system has led us to great success in competitions as well. In the 2017 Chesapeake District Championships event, our team was ranked 52nd out of 58 teams, and yet was chosen 12th in the elimination rounds because we showed top-ranking teams about our real capabilities and made it to the 3rd round of grand finals, where one alliance member disconnected and our robot got stuck on a gear. As an alliance captain at girlPOWER in 2018, we chose the 13th and 14th ranked teams (out of 14 teams) due to the results we saw in our data, and went on to win the event.