[robocup-small] November RoboCup SSL Update and Response to Request of Transparency

[RoboTeam Twente]

Dear SSL-Community and Committee members,

We would like to start this message by thanking you for consulting the
community on the proposed rule changes and that we are saddened by the
sudden resignation of our beloved OC members. Like the teams that gave
their feedback before us, we are concerned about the amount and impact of
the rule changes this late in the year. We evaluated the proposed changes
disregarding the difficulties that may occur in the implementation of our
software, to increase the quality of the SSL.

*Number of robots + field size*
We agree with the increase of robots from 8 to 11, as was determined in the
open TC/OC meeting. Like ER-Force, we do not think that this increment is
feasible without increasing the field size. The reasoning is that the
current field size causes the same area per robot as in Division B. While
this argumentation seems reasonable, there are some problems. The type of
gameplay differs (greatly) between the two divisions. In Division A, the
gameplay is more dynamic and defence is a primary focus. Maintaining the
current field size could cause more crashes or decrease the speed of
gameplay in Division A, slowing down the games. Furthermore, as ER-Force
pointed out in their email, it is possible to cover the entire goal line
using 11 robots.

We propose that in order to work with 11 robots, the field size should also
be increased. If the financial resources to increase the field size are not
available, we should work with 8 robots in Bordeaux.

*Introducing walls into a subset of games (DivA only)*
While we understand the motivation behind adding walls to the games, we do
not support the initiative. We think it is a treatment for the symptom,
when the underlying problem is that we simply need to get better at keeping
the ball in the field. By introducing handicaps like walls, we are suddenly
playing an entirely different game. This diverges from the original goal of
the robocup, where the SSL was meant to focus on creating the top-level
strategies, which can then be translated to the middle size and humanoid
league, to eventually accomplish the goal of creating a humanoid soccer
team.

More specifically, adding walls to the field opens up multiple new
strategies and tactics that all do not work for real soccer. Therefore, we
essentially produce research that cannot easily be applied to the other
leagues.

Finally, should the TC/OC still consider it necessary to add walls, we
would propose they only be placed at the sides of the field, and not at the
goal lines. Taking a shot at goal should be risky: in real soccer, if you
take a shot at goal and miss, you lose your ball possession and have to
start your attack all over again. However, with the currently proposed
changes, as long as you are fast enough, you can send a lot of robots close
to the goal, and they will just reflex-kick continuously in the direction
of the defending goal, until the defending team can chip/clear the ball to
the other side of the field. Such a scenario is more of a shooting gallery
than a soccer match.

*Merge indirect and direct free kick into direct free kick (both Div)*
The difference between an indirect and direct free kick is difficult to
detect for the autoRef as well as the human referee. This casued several
arguments during the matches and until we can properly detect when a free
kick should be direct or indirect, we agree that both should be merged into
direct free kick. We do believe that the autoRef should be improved to
properly detect the difference, since direct and indirect free kicks are
used in human soccer games which means we have to be able to work with them
some day.

*Remove the double touch foul (both Div)*
We agree with the removal of double touch, since this will encourage the
teams to experiment with dribbling by small kicks. We do however wish that
the double touch foul remains valid during penalties (if they are not
removed) since this allows for unfair gameplay (and is also not allowed in
real soccer).


*Robot interchange during game play (DivA only) + Do not stop game for
yellow cards (both Div)*The way we understand the current changes, we are
afraid that it might get unsafe for the person that has to interchange the
robots. Also, with the cards not bound to individual robots, the software
does not know which robot the team wants to take out of the field. If this
was known, the robot could drive to a position on the field where it can be
safely removed. The way we understand it, someone has to now remove a
moving robot from the field. Furthermore, the outfit of an individual could
mess with the vision and the individual can be detected as a ball.

As a final note we would like to see a clear goal for the SSL. With the
proposed changes we are getting further and further away from an actual
soccer game, which is the current goal, and working towards a game which is
'fun to look at'.

On behalf of RoboTeam Twente,

Met vriendelijke groet / Kind regards,

Selina Zwerver

*Finance & Logistics Manager*
*RoboTeam Twente | University of Twente*

Mobiel:    +31 618 70 2636
Internet:   RoboTeam Twente <https://roboteamtwente.nl/>
LinkedIn:  Selina Zwerver
<https://www.linkedin.com/in/selina-zwerver-5462bb129/>



On Mon, 25 Nov 2019 at 18:34, Lukas Wegmann <
lukas.wegmann at robotics-erlangen.de> wrote:

> Dear SSL-Community and Committee members,
>
> first of all we would like to thank you for being open about the upcoming
> rule proposals and enabling feedback.
> That being said, we have various concerns and comments regarding them.
>
> We wish to preface this by saying that, while we appreciate striving to
> improve the SSL as a whole, the proposed changes are numerous, sudden and
> time intensive to implement. The consequence will be that less time can be
> spent on developing actual AI improvements that would contribute to
> meaningful research. This is reinforced by the fact that many if not most
> of the changes are completely unexpected and some even directly contradict
> decisions made in the open TC/OC meeting of the previous RoboCup, which we
> find especially unacceptable.
>
> Nonetheless we wish to provide constructive feedback on the proposals with
> the intent of contributing to make the SSL as best as it can be.
>
>
> *Number of robots increased to 11 (DivA only)*
> This change was already agreed upon during the open TC/OC meeting and we
> think that it will improve gameplay and make further research necessary.
> However, without the increase of the field size, it is not clear if this
> change has the desired effects. The argument that there will still be more
> space per robot than in Division B is lacking as no one said that Division
> B has the optimal state. Especially since the defense of Division A teams
> tends to be better than most Division B teams, even fewer goals could be
> shot out of regular game play.
>
> This fact can be shown by the image in the attachment, showing 11 robots
> acting as defenders. Although there is some space between the robots, they
> can essentially cover the whole front side of the defense area, which would
> make it very hard to score goals against a defensive team.
>
> Furthermore, we believe the statement that an increased field size would
> not have any significant effect on ball outs to be vastly incorrect. This
> years final game serves as a fine example of this, when the ball did not
> leave the field for minutes at a time, leading to the referee having to
> manually end  the match over a minute after the official game time was
> over. While this is in general dependent on the teams in question, the
> possibility of keeping the ball in play longer should serve as an incentive
> for improving ball control and we think that keeping the field size as it
> is would mitigate that.
> Introducing walls into a subset of games (DivA only)
> While in general we are not opposed to introducing walls to the SSL
> league, we find it quite late in the year to implement such a radical
> change. This would necessitate major changes in our software to properly
> use the new options in the offense as well as to properly defend against an
> attacker using such a strategy. We would prefer to have such radical
> changes discussed in person at the RoboCup instead of them suddenly being
> announced via email.
>
> *Reduce stoppages due to fouls (both Div)*
> The proposed changes on how to handle fouls seem inconsistent. First of
> all, some offenses are explicitly marked to increase the foul counter while
> some others are not. Continuing the game after excessive dribbling while
> still counting goals shot after the fact allows teams to gain a huge
> advantage in exchange for a relatively minor punishment. Giving Force Start
> instead of a Freekick for some fouls, e.g. Kick Timeout or Defender too
> close to Ball, without increasing the foul counter effectively removes
> these rules. Also, "the percentage of game-states" does not necessary
> correlate with "spent time on solving the problem" as stated in the long
> term goals. We rarely focus our development time on non-game-states, as we
> cannot score during those. While it still might be an interesting metric
> for OC and viewers, it does not reflect the time spent to solve a
> particular problem, and therefore should not be overvalued by rule changes.
> For further comments, see the Google Document.
>
> *Add a cost to timeouts (DivA only)*
> While we can agree that reducing stoppage time of any kind, we have
> several issues with this rule change in particular.
> First of all we believe that inventing arbitrary and unrelated game
> punishments to discourage a certain team behavior is lackluster at best. So
> far there is not a single case where a certain team behaviour (except for
> severe unsporting behavior) has any effect on the game state and we think
> it should stay that way. Freekicks and fouls are designed for the AI and
> robots, not their creators. There are plenty of other ways to discourage or
> reduce timeouts and since many of the following rule changes can already
> cost timeouts, we believe that no further punishment is necessary.
>
> *Remove Penalty Kicks (both Div)*
> While we found the number of penalty kicks during the RoboCup unreasonable
> (even given that we won most of our games due to penalty kicks), the
> proposed change to drop them entirely might result in adverse effects. For
> one, this punishes Division B teams far more than Division A teams. When
> having 11 robots, losing one, two or even three is not such a hard
> punishment, while when the game is played with only 6 robots, losing three
> is catastrophic. Punishing Division B teams more for fouls is not
> reasonable as higher standards regarding fouls should be applied to
> Division A teams instead.
>
> Additionally, in case a Division A team does not have 11 robots (which
> will most likely happen for some teams next year) or the game has only a
> few minutes left, red and yellow cards mostly lose their significance.
> Therefore, red cards may not present a sufficient punishment in Division A.
>
> *Vision dropouts (both Div)*
> While it is possible to navigate the own robots even with severely
> degraded vision information, nothing can be done about the opponent robots
> or the ball. Losing too much vision information about the opponent will
> result in additional collisions and other fouls. In addition, the autoref
> will not be able to properly judge the situation without the required level
> of vision fidelity.
> Having the teams make sure the vision is good enough before the game will
> not solve this problem as games are often played directly one after the
> other, leaving no time for vision adjustments in between.
> Offering incentives to improve the SSL vision software might solve the
> underlying problem in a more efficient and sustainable manner.
>
>
> * Final thoughts*
> Many of the proposed changes are just too much and too late after the
> RoboCup. Such big rule changes should be discussed after the RoboCup during
> the open TC/OC meeting and the conclusions reached there should at least
> roughly be honored.
> While many of the changes would surely succeed in bringing the non-game
> time during games down to a minimum, they do not present significant
> incentive or reason for more research in our league and might even hinder
> it due to the time drain of additional implementation work. Building robots
> is a slow, exhaustive and expensive process. A lot of teams rely on the
> fact that their robots can be used for some time until the financial and
> human resources recover.
>
> This fact does lead me to wonder weather "big changes with rapid adoption"
> are good for the league. It could lead to cheaper robots with shorter
> lifetime, decreasing the overall performance of the league and risking our
> position as the "most interesting league to watch on the Robocup"
>
> Therefore, we hope that the changes are reconsidered and properly
> discussed in Bordeaux after the RoboCup.
>
> Unrelated to the rule changes, we still miss a proper response and
> clarification regarding the recent events like the Thunderbots have already
> asked for. The resignation of two cherished OC members is unprecedented and
> the email exchange leading up to this raises numerous concerns about the
> way business is conducted.  We wholeheartedly back the call for information
> that Thunderbots initiated. We are in desperate need of more transparency.
>
>
> With best regards
>
> ER-Force
> On 20.11.19 06:51, Carla Cosenza wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> * Dear SSL Community, We have been striving for a long time to increase
> actual soccer game-on time during SSL games. Unfortunately, despite several
> rules iterations, our analysis over the game logs from RoboCup 2018 and
> 2019 show that we are still far off the mark - games had an average of only
> 20% actual game-on time during RoboCup 2018, and 15% actual game on time
> during RoboCup 2019. To make the situation worse, we have far more special
> game states than game on. All the recent events have been caused by the
> lack of established procedure for making big changes with rapid adoption
> for the Small Size League. We were trying to decrease the amount of time
> not spent on actual soccer during matches and did not manage to come to an
> agreement. In order to prevent this, we are creating a method for these
> decisions. Due to its importance, it will be discussed in Bordeaux with
> representation of all teams and committees. For RoboCup 2020, we have
> created a proposal for rule changes:
> https://docs.google.com/document/d/1EBfKZfXEYq2SpcLzUb7OE68vlUKjJnk_tnaulR5CMO8
> <https://docs.google.com/document/d/1EBfKZfXEYq2SpcLzUb7OE68vlUKjJnk_tnaulR5CMO8>
> You can also find an immutable copy of the document attached to this mail.
> We encourage teams to give us feedback by the end of November. You can use
> the comment and suggestion mode or reply to this mail. If there is well
> justified concern, we’ll consider it when finalizing the rule changes. Some
> of the most notable changes are: Walls for field boundaries in some Div A
> matches In order to verify the possibility of removing the out of bounds
> part of the field, we are going to test, in some division A matches, having
> walls on the boundaries of the field. This will result in having less game
> states and less time spent placing the ball after it leaves the field. Each
> team will be required to play at least one game with the walls during the
> group stage and before the match starts, both teams need to opt out of the
> walls in order not to use them. The objective of this change is to reduce
> the amount of time spent not playing soccer during the matches. Our
> statistical analysis has shown that games had an average of only 20% actual
> game-on time during RoboCup 2018, and 15% actual game on time during
> RoboCup 2019. 11 robots and field size for Div A Having Div A reach 11
> robots has been on the SSL roadmap for this year and it will happen. Div A
> teams will be allowed to have up to 11 robots on field. The only difference
> is that there will not be an increase in field size. We have measured that
> even if we do not change the field size, taking Div A to 11 robots will
> provide a bigger area per robot value (4.9m²) than the current
> configuration for Div B does (4.5m²). Match Statistics When discussing rule
> changes, we need some facts and numbers from past RoboCups to support or
> reject arguments. We also need to evaluate the effects of changes that were
> made. That’s why we started to establish a standard statistics database.
> The SSL committee commits on updating and evaluating these statistics each
> year. The code can be found here:
> https://github.com/RoboCup-SSL/ssl-match-stats
> <https://github.com/RoboCup-SSL/ssl-match-stats> We will prepare an area on
> the SSL website with more information on this soon. Qualification
> requirements We will update the specifications for the qualification videos
> for Div A. Team have to demonstrate passing (with a more precise
> definition) and ball placement. Both can still be demonstrated by game play
> footage, but we will limit the duration of the video for both divisions.
> This is to encourage teams to work on smooth game play. Details will be
> included in the qualification webpage
> <https://ssl.robocup.org/robocup-2020-qualification/>. Technical challenges
> We will repeat a version of the SSL-Vision Blackout Challenge. Both
> divisions are eligible to participate. We will also have a Ball Placement
> Challenge for Div B teams only. All Div B teams will be required to
> participate in the Ball Placement Challenge. The intention is to prepare
> Div B teams to move to Div A. More information about these challenges will
> be announced later. The current work-in-progress status can be found on
> GitHub: https://github.com/RoboCup-SSL/technical-challenge-rules
> <https://github.com/RoboCup-SSL/technical-challenge-rules>.  Other
> reminders and updates - As stated in the previous email, all teams will be
> required to bring posters in order to present their changes and
> improvements to the other teams. Like last year we will schedule a poster
> session where team members can present and discuss poster contents. - We
> are still testing April Tags for SSL Vision. Initial implementation works
> correctly but more testing is needed. We do not anticipate forcing teams to
> use the April tags at this competition. However teams should start
> preparing for tags other than the currently butterfly patterns.  All of
> these changes were based on the long term goals of the Small Size League.
> The main intended effect is to:  Spend more time solving the game-on
> regular soccer problem, not on non-soccer behaviors.  In particular, we
> anticipate ball handling (stealing, possession, dribbling, passing) to be
> more important now. We are also trying to propose challenges to help Div B
> teams reach Div A, since Div B is intended to simplify the entrance of a
> new team to the league. Team Leader Online Meeting We would also like to
> invite the team leaders for an online meeting in order to clarify any
> concerns and discuss the decisions for RoboCup 2020. We ask each team to
> fill out this form
> (https://docs.google.com/forms/d/e/1FAIpQLScXR8BGCPYzZVHh0YUw8S75rYmV5HmCZk6xdenIvRXbV5yhbQ/viewform?usp=sf_link
> <https://docs.google.com/forms/d/e/1FAIpQLScXR8BGCPYzZVHh0YUw8S75rYmV5HmCZk6xdenIvRXbV5yhbQ/viewform?usp=sf_link>)
> with the name and email of their team leader so we can send out the invite
> for the meeting.  We look forward to hearing from you! Thank you! The
> Technical, Organization, and Executive Committees of the Small Size League *
>
> _______________________________________________
> robocup-small mailing listrobocup-small at cc.gatech.eduhttps://mailman.cc.gatech.edu/mailman/listinfo/robocup-small
>
> --
> Lukas Wegmann
> Vorstand / Chairman
> Robotics Erlangen e. V.
> lukas.wegmann at robotics-erlangen.de
> http://www.robotics-erlangen.de
> _______________________________________________
> robocup-small mailing list
> robocup-small at cc.gatech.edu
> https://mailman.cc.gatech.edu/mailman/listinfo/robocup-small
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/robocup-small/attachments/20191127/d5790254/attachment-0001.html>