[robocup-worldwide] multisensor datasets from project Rawseeds are online

the Rawseeds project rawseeds at elet.polimi.it
Wed Jul 1 05:56:02 EDT 2009


[apologies if you receive this more than once]

Dear all,

the Rawseeds project (http://www.rawseeds.org/) is an EU-funded effort
aimed at building a high-quality "Benchmarking Toolkit" for Simultaneous
Localization And Mapping. The first element of the Toolkit, i.e., a set
of 11 multisensor datasets, can be reached from:

  http://www.rawseeds.org/home/category/benchmarking-toolkit/datasets/

All of the datasets include data from 4 LRF, a trinocular stereo rig, a 
panoramic camera, an IMU, sonars (indoors only), a low-cost monocular
camera, and classical odometry.

Rawseeds' datasets cover indoor, outdoor and mixed environments. Each of
them includes the full output of the sensors during an uninterrupted
exploration session 20+ minutes long, plus suitable ground truth data
covering part of the explored areas: RTK GPS for outdoor datasets, and
~10cm-precision data in a "ground truth area" for indoor datasets.
Sensor streams and ground truth data are time-synchronized and have been
subjected to extensive validation.

In a few weeks the second part of the Benchmarking Toolkit will be
published. It is a set of problems, called "Benchmark Problems", which
use our datasets as their input. The key point is that each BP includes
a set of precisely defined *performance metrics*, to be used to evaluate
the output of any algorithm which solves the BP. The description of such
an algorithm, together with the value of the performance metrics and
other associated information, is called a "Benchmark Solution".
Rawseeds' BPs are presently focused on SLAM, but other types of problems
might be proposed in the future.

The third - and final - part of the Benchmarking Toolkit will be a set
of Benchmark Solutions based on state-of-the-art algorithms. These are
intended as examples and references for users: in fact, as soon as they
are online, Rawseeds will start asking the robotics community for the
submission of new BSs. Any algorithm which complies with the
specifications of a BS will be published along with the other BSs.

The key point is this: as all BSs associated to the same BP include
their own values for the performance metrics defined by the BP, so
*benchmarking of algorithms for autonomous robotics, and quantitative
comparison between different algorithms, will become feasible*. This
is, indeed, the goal of Rawseeds.

Publishing an algorithm (in the form of a BS) on Rawseeds' website
will be a good way to give it visibility *and*, at the same time,
will make its performances immediately and quantitatively comparable
with those of established algorithms. So we hope to quickly gather a
good number of users/contributors. By the way, transforming an
existing algorithm into a BS is a straightforward process; and a
flexible Intellectual Property Management system has been defined.

More information about Rawseeds and its Benchmarking Toolkit can of
course be found on the project's website,

  http://www.rawseeds.org/

along with a contact form, a forum, and more. We hope to see you there!

The Rawseeds Consortium



More information about the robocup-worldwide mailing list