[robocup-small] November RoboCup SSL Update and Response to Request of Transparency

[Lukas Wegmann]

Dear SSL-Community and Committee members,

first of all we would like to thank you for being open about the 
upcoming rule proposals and enabling feedback.
That being said, we have various concerns and comments regarding them.

We wish to preface this by saying that, while we appreciate striving to 
improve the SSL as a whole, the proposed changes are numerous, sudden 
and time intensive to implement. The consequence will be that less time 
can be spent on developing actual AI improvements that would contribute 
to meaningful research. This is reinforced by the fact that many if not 
most of the changes are completely unexpected and some even directly 
contradict decisions made in the open TC/OC meeting of the previous 
RoboCup, which we find especially unacceptable.

Nonetheless we wish to provide constructive feedback on the proposals 
with the intent of contributing to make the SSL as best as it can be.


_*Number of robots increased to 11 (DivA only)*_
This change was already agreed upon during the open TC/OC meeting and we 
think that it will improve gameplay and make further research necessary.
However, without the increase of the field size, it is not clear if this 
change has the desired effects. The argument that there will still be 
more space per robot than in Division B is lacking as no one said that 
Division B has the optimal state. Especially since the defense of 
Division A teams tends to be better than most Division B teams, even 
fewer goals could be shot out of regular game play.

This fact can be shown by the image in the attachment, showing 11 robots 
acting as defenders. Although there is some space between the robots, 
they can essentially cover the whole front side of the defense area, 
which would make it very hard to score goals against a defensive team.

Furthermore, we believe the statement that an increased field size would 
not have any significant effect on ball outs to be vastly incorrect. 
This years final game serves as a fine example of this, when the ball 
did not leave the field for minutes at a time, leading to the referee 
having to manually end  the match over a minute after the official game 
time was over. While this is in general dependent on the teams in 
question, the possibility of keeping the ball in play longer should 
serve as an incentive for improving ball control and we think that 
keeping the field size as it is would mitigate that.
Introducing walls into a subset of games (DivA only)
While in general we are not opposed to introducing walls to the SSL 
league, we find it quite late in the year to implement such a radical 
change. This would necessitate major changes in our software to properly 
use the new options in the offense as well as to properly defend against 
an attacker using such a strategy. We would prefer to have such radical 
changes discussed in person at the RoboCup instead of them suddenly 
being announced via email.

_*Reduce stoppages due to fouls (both Div)*_
The proposed changes on how to handle fouls seem inconsistent. First of 
all, some offenses are explicitly marked to increase the foul counter 
while some others are not. Continuing the game after excessive dribbling 
while still counting goals shot after the fact allows teams to gain a 
huge advantage in exchange for a relatively minor punishment. Giving 
Force Start instead of a Freekick for some fouls, e.g. Kick Timeout or 
Defender too close to Ball, without increasing the foul counter 
effectively removes these rules. Also, "the percentage of game-states" 
does not necessary correlate with "spent time on solving the problem" as 
stated in the long term goals. We rarely focus our development time on 
non-game-states, as we cannot score during those. While it still might 
be an interesting metric for OC and viewers, it does not reflect the 
time spent to solve a particular problem, and therefore should not be 
overvalued by rule changes. For further comments, see the Google Document.

_*Add a cost to timeouts (DivA only)*_

While we can agree that reducing stoppage time of any kind, we have 
several issues with this rule change in particular.
First of all we believe that inventing arbitrary and unrelated game 
punishments to discourage a certain team behavior is lackluster at best. 
So far there is not a single case where a certain team behaviour (except 
for severe unsporting behavior) has any effect on the game state and we 
think it should stay that way. Freekicks and fouls are designed for the 
AI and robots, not their creators. There are plenty of other ways to 
discourage or reduce timeouts and since many of the following rule 
changes can already cost timeouts, we believe that no further punishment 
is necessary.

_*Remove Penalty Kicks (both Div)*_

While we found the number of penalty kicks during the RoboCup 
unreasonable (even given that we won most of our games due to penalty 
kicks), the proposed change to drop them entirely might result in 
adverse effects. For one, this punishes Division B teams far more than 
Division A teams. When having 11 robots, losing one, two or even three 
is not such a hard punishment, while when the game is played with only 6 
robots, losing three is catastrophic. Punishing Division B teams more 
for fouls is not reasonable as higher standards regarding fouls should 
be applied to Division A teams instead.

Additionally, in case a Division A team does not have 11 robots (which 
will most likely happen for some teams next year) or the game has only a 
few minutes left, red and yellow cards mostly lose their significance. 
Therefore, red cards may not present a sufficient punishment in Division A.


_*Vision dropouts (both Div)*_
While it is possible to navigate the own robots even with severely 
degraded vision information, nothing can be done about the opponent 
robots or the ball. Losing too much vision information about the 
opponent will result in additional collisions and other fouls. In 
addition, the autoref will not be able to properly judge the situation 
without the required level of vision fidelity.
Having the teams make sure the vision is good enough before the game 
will not solve this problem as games are often played directly one after 
the other, leaving no time for vision adjustments in between.
Offering incentives to improve the SSL vision software might solve the 
underlying problem in a more efficient and sustainable manner.

_*
*__*Final thoughts*_
Many of the proposed changes are just too much and too late after the 
RoboCup. Such big rule changes should be discussed after the RoboCup 
during the open TC/OC meeting and the conclusions reached there should 
at least roughly be honored.
While many of the changes would surely succeed in bringing the non-game 
time during games down to a minimum, they do not present significant 
incentive or reason for more research in our league and might even 
hinder it due to the time drain of additional implementation work. 
Building robots is a slow, exhaustive and expensive process. A lot of 
teams rely on the fact that their robots can be used for some time until 
the financial and human resources recover.

This fact does lead me to wonder weather "big changes with rapid 
adoption" are good for the league. It could lead to cheaper robots with 
shorter lifetime, decreasing the overall performance of the league and 
risking our position as the "most interesting league to watch on the 
Robocup"

Therefore, we hope that the changes are reconsidered and properly 
discussed in Bordeaux after the RoboCup.

Unrelated to the rule changes, we still miss a proper response and 
clarification regarding the recent events like the Thunderbots have 
already asked for. The resignation of two cherished OC members is 
unprecedented and the email exchange leading up to this raises numerous 
concerns about the way business is conducted.  We wholeheartedly back 
the call for information that Thunderbots initiated. We are in desperate 
need of more transparency.


With best regards

ER-Force

On 20.11.19 06:51, Carla Cosenza wrote:
> *
>
> Dear SSL Community,
>
>
> We have been striving for a long time to increase actual soccer 
> game-on time during SSL games. Unfortunately, despite several rules 
> iterations, our analysis over the game logs from RoboCup 2018 and 2019 
> show that we are still far off the mark - games had an average of only 
> 20% actual game-on time during RoboCup 2018, and 15% actual game on 
> time during RoboCup 2019. To make the situation worse, we have far 
> more special game states than game on.
>
>
> All the recent events have been caused by the lack of established 
> procedure for making big changes with rapid adoption for the Small 
> Size League. We were trying to decrease the amount of time not spent 
> on actual soccer during matches and did not manage to come to an 
> agreement. In order to prevent this, we are creating a method for 
> these decisions. Due to its importance, it will be discussed in 
> Bordeaux with representation of all teams and committees.
>
>
> For RoboCup 2020, we have created a proposal for rule changes: 
> https://docs.google.com/document/d/1EBfKZfXEYq2SpcLzUb7OE68vlUKjJnk_tnaulR5CMO8
>
> You can also find an immutable copy of the document attached to this 
> mail. We encourage teams to give us feedback by the end of November. 
> You can use the comment and suggestion mode or reply to this mail.
>
> If there is well justified concern, we’ll consider it when finalizing 
> the rule changes. Some of the most notable changes are:
>
>
> Walls for field boundaries in some Div A matches
>
> In order to verify the possibility of removing the out of bounds part 
> of the field, we are going to test, in some division A matches, having 
> walls on the boundaries of the field. This will result in having less 
> game states and less time spent placing the ball after it leaves the 
> field. Each team will be required to play at least one game with the 
> walls during the group stage and before the match starts, both teams 
> need to opt out of the walls in order not to use them. The objective 
> of this change is to reduce the amount of time spent not playing 
> soccer during the matches. Our statistical analysis has shown that 
> games had an average of only 20% actual game-on time during RoboCup 
> 2018, and 15% actual game on time during RoboCup 2019.
>
>
> 11 robots and field size for Div A
>
> Having Div A reach 11 robots has been on the SSL roadmap for this year 
> and it will happen. Div A teams will be allowed to have up to 11 
> robots on field. The only difference is that there will not be an 
> increase in field size. We have measured that even if we do not change 
> the field size, taking Div A to 11 robots will provide a bigger area 
> per robot value (4.9m²) than the current configuration for Div B does 
> (4.5m²).
>
>
> Match Statistics
>
> When discussing rule changes, we need some facts and numbers from past 
> RoboCups to support or reject arguments. We also need to evaluate the 
> effects of changes that were made. That’s why we started to establish 
> a standard statistics database. The SSL committee commits on updating 
> and evaluating these statistics each year.
>
> The code can be found here: https://github.com/RoboCup-SSL/ssl-match-stats
>
> We will prepare an area on the SSL website with more information on 
> this soon.
>
>
> Qualification requirements
>
> We will update the specifications for the qualification videos for Div 
> A. Team have to demonstrate passing (with a more precise definition) 
> and ball placement. Both can still be demonstrated by game play 
> footage, but we will limit the duration of the video for both 
> divisions. This is to encourage teams to work on smooth game play. 
> Details will be included in the qualification webpage 
> <https://ssl.robocup.org/robocup-2020-qualification/>.
>
>
> Technical challenges
>
> We will repeat a version of the SSL-Vision Blackout Challenge. Both 
> divisions are eligible to participate. We will also have a Ball 
> Placement Challenge for Div B teams only. All Div B teams will be 
> required to participate in the Ball Placement Challenge.The intention 
> is to prepare Div B teams to move to Div A. More information about 
> these challenges will be announced later. The current work-in-progress 
> status can be found on GitHub: 
> https://github.com/RoboCup-SSL/technical-challenge-rules.
>
>
> Other reminders and updates
>
>  *
>
>     As stated in the previous email, all teams will be required to
>     bring posters in order to present their changes and improvements
>     to the other teams. Like last year we will schedule a poster
>     session where team members can present and discuss poster contents.
>
>  *
>
>     We are still testing April Tags for SSL Vision. Initial
>     implementation works correctly but more testing is needed. We do
>     not anticipate forcing teams to use the April tags at this
>     competition. However teams should start preparing for tags other
>     than the currently butterfly patterns.
>
>
> All of these changes were based on the long term goals of the Small 
> Size League. The main intended effect is to:
>
> Spend more time solving the game-on regular soccer problem, not on 
> non-soccer behaviors.
>
> In particular, we anticipate ball handling (stealing, possession, 
> dribbling, passing) to be more important now. We are also trying to 
> propose challenges to help Div B teams reach Div A, since Div B is 
> intended to simplify the entrance of a new team to the league.
>
>
> Team Leader Online Meeting
>
> We would also like to invite the team leaders for an online meeting in 
> order to clarify any concerns and discuss the decisions for RoboCup 
> 2020. We ask each team to fill out this form 
> (https://docs.google.com/forms/d/e/1FAIpQLScXR8BGCPYzZVHh0YUw8S75rYmV5HmCZk6xdenIvRXbV5yhbQ/viewform?usp=sf_link) 
> with the name and email of their team leader so we can send out the 
> invite for the meeting.
>
>
> We look forward to hearing from you!
>
>
> Thank you!
>
>
> The Technical, Organization, and Executive Committees of the Small 
> Size League
>
> *
>
> _______________________________________________
> robocup-small mailing list
> robocup-small at cc.gatech.edu
> https://mailman.cc.gatech.edu/mailman/listinfo/robocup-small
-- 
Lukas Wegmann
Vorstand / Chairman
Robotics Erlangen e. V.
lukas.wegmann at robotics-erlangen.de 
<\"mailto:lukas.wegmann at robotics-erlangen.de\">
http://www.robotics-erlangen.de <\"http://www.robotics-erlangen.de\">
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/robocup-small/attachments/20191125/a1152436/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 11-robot-defense-area.png
Type: image/png
Size: 14003 bytes
Desc: not available
URL: <https://mailman.cc.gatech.edu/pipermail/robocup-small/attachments/20191125/a1152436/attachment-0001.png>