SecondHands at TAROS 2019

The SecondHands project has been presented at the 20th Towards Autonomous Robotic Systems (TAROS) 2019 Conference, at Queen Mary University of London, from the 3rd to the 5th of July 2019.

The conference, hosted by the Centre for Advanced Robotics at Queen Mary (ARQ), presented contributions in several areas of robotics, from soft robotics to locomotion, planning and grasping, and it is an important even within the UK. Among the keynote speakers was Prof. Aude Billard, which is also one of the principal investigators of the Secondhands project and presented some of the scientific achievements of the project.

The results of Secondhands were showcased at a dedicated booth, where staff if Ocado Technology described the aim and vision of the project and the recent achievements of the consortium. The booth attracted good interest from the audience as an average of 20 – 25 people discussed the scientific achievements and the possible exploitation of the project.

secondhands taros



Secondhands at the 8th bi-annual Joint Action Meeting (JAM)

EPFL will present the Secondhands project at the 8th bi-annual Joint Action Meeting (JAM), which will be held at the Italian Institute of Technology, Genova, Italy, on 10-13 July 2019.

Dr. Mahdi Khoramshahi has been invited to present the achievements of the SecondHands consortium in the talk Intention recognition using state dependent dynamical system in physical human-robot joint action on 12 July 2019.

The JAM conference brings together cognitive scientists and researchers from related disciplines studying individuals’ ability to act together.


Secondhands at the RSS 2019 Workshop on assistive technologies

EPFL presented the Secondhands project at AI and Its Alternatives in Assistive and Collaborative at the Robotics RSS Workshop on 23th of June 2019.

Prof. Aude Billard and Dr. Mahdi Khoramshahi were invited to present the achievements of the Secondhands consortium in the talk From human-intention recognition to compliant control. The topic of the talk was the following:

Human ability to coordinate one’s actions with other individuals to perform a task together is fascinating. For example, we coordinate our action with others when we carry a heavy object or when we construct a piece of furniture. Capabilities such as (1) force/compliance adaptation, (2) intention recognition, and (3) action/motion prediction enables us to assist others and fulfill the task. For instance, by adapting the compliance, we not only reject undesirable perturbations that undermine the task but also incorporate others’ motions into the interaction. Complying with partners’ motions allows us to recognise their intention and consequently predict their actions. With the growth of factories involving humans and robots working side by side, designing controllers and algorithms with such capacities is a crucial step toward assistive robotics. The challenge, however, is to attain a unified control strategy with predictive and adaptive capacities at the task, motion, and force-level which ensures a stable and safe interaction. In this talk, we present a state-dependent dynamical system-based approach for prediction and control in physical human-robot interactions.

The workshop aimed at bringing together researchers in autonomous systems and in psychology, neuroscience and bio-mechanics to better understand how a human partner thinks and functions when interacting with artificial agents.


SecondHands sponsors workshop at IROS 2018: The intelligence of touch

The SecondHands project will be sponsoring the “The intelligence of touch” workshop at IROS 2018. The workshop will be held on the 1st of October 2018 in Madrid.

The aim of the workshop is exploring how tactile feedback in the wider sense can be exploited by creating sensors, robots and intelligence which take full advantage of this information and how humans are processing, interpreting and reacting on tactile cues. Submissions for the workshop are open.

Prof. Tamim Asfour is one of the organisers of the workshop which features internationally renowned robotic leaders from industry and academia, such as Prof. Aude Billard (EPFL), head of the Learning Algorithms and Systems Laboratory (LASA) at Ecole Polytechnique Federale de Lausanne (EPFL) and PI of the SecondHands project for EPFL.

The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) is one of the most prestigious annual academic conferences in robotics. The conference, which started in 1988, has reached its 31st year and will be held in Madrid, Spain, from the 1st to the 5th of October 2018.


The SecondHands project at CeBIT 2018

The SecondHands project has been showcased at the CeBIT technology exhibition, in Hannover, Germany, from the 11th to the 15th of June.

The SecondHands stand attracted a lot of interest from visitors and the local media. The team at KIT performed more than 50 demonstrations of the SecondHands robot, ARMAR-6, helping a human technician in doing maintenance work of Ocado’s highly automated warehouses (Customer Fulfillment Centre).

The SecondHands project team demonstrated the humanoid robot ARMAR-6 collaborating with a human worker in a bimanual overhead task and performing force-based bimanual manipulation, vision-based grasping, fluent object handover, human activity recognition, natural language based human-robot dialog and interaction, navigation among many other features. The use case shown in the demonstrations was supplied by Ocado. In a complex maintenance task demonstration, the robot was able to recognise the need of help of a technician based on speech, force and visual information.  The demonstrations were filmed by national and international televisions and was ranked as one of the top CeBIT 2018 highlights.


The CeBIT exhibition is one of the largest and most representative technological exhibitions in the world. It covers hot topics and trends in digital technology such as AI, robotics, Internet of Things and big data among others. It covers the digitisation of business, government and society from every angle. In 2017, more than 200,000 people attended CeBIT.


Other press resources on the exhibition are available at this and this link in German.


SecondHands project members present first robot prototype ARMAR-6

Today the SecondHands project is presenting the first prototype of its collaborative robot which will act as the main platform for testing and developing new technologies related to the maintenance and repair of automation equipment in Ocado’s highly automated warehouses using a robot assistant.

The SecondHands robot prototype ARMAR-6 has been developed at the Karlsruhe Institute of Technology (KIT) by Tamim Asfour and his team at the High Performance Humanoid Technologies Lab (H²T) of the Institute for Anthropomatics and Robotics.

SecondHands is an EU-funded Horizon 2020 project aiming to design a collaborative robot (cobot) that can proactively offer support to maintenance technicians working in Ocado’s highly automated warehouses, also known as Customer Fulfilment Centers (CFCs). This robot will be a second pair of hands that will assist technicians when they are in need of help. The robot will learn through observation and will augment the humans’ capabilities by completing tasks that require a level of precision or physical strength that are not available to human workers.

The SecondHands robot prototype has been delivered to the Ocado Technology robotics research lab where experiments to evaluate the integrated research components from all project partners is currently taking place. The video below presents the first instances of the robot interacting with its testing environment right after it was assembled:


The SecondHands project combines the skills of world class researchers focusing on a real-world industrial use case to deliver:

  • the design of a new robotic assistant
  • a knowledge base to facilitate proactive help
  • a high degree of human-robot interaction
  • advanced perception skills to function in a highly dynamic industrial environment

Together with its research partners École Polytechnique Fédérale de Lausanne (EPFL), Karlsruhe Institute of Technology (KIT), Sapienza Università di Roma, and University College London (UCL), Ocado Technology is working to advance the technology readiness of areas such as computer vision and cognition, human-robot interaction, mechatronics, and perception and ultimately demonstrate how versatile and productive human-robot collaboration can be in practice. Here is a summary of the research contributions for each of the project partners:

  • EPFL: human-robot physical interaction with bi-manipulation, including action skills learning
  • KIT (H²T): Development of the ARMAR-6 robot including its entire mechatronics, software operating system and control as well as robot grasping and manipulation skills.
  • KIT (Interactive Systems Lab, ISL): the spoken dialog management system
  • Sapienza University of Rome: visual scene perception with human action recognition, cognitive decision making, task planning and execution with continuous monitoring
  • UCL: computer vision techniques for 3D human pose estimation and semantic 3D reconstruction of dynamic scenes
  • Ocado Technology: integration of researched functionality on the robot platform and evaluation in real-world demonstrations

Collaborative robots represent a fast-growing segment of the industrial robots market. According to the World Robotics Report released earlier this year by the International Federation of Robotics (IFR), industrial robot installations are forecast to grow by 15% in 2018. This increased adoption of robots for a wide range of applications comes off the back of stronger-than-expected growth in the global economy, faster business cycles, greater variety in customer demand, and the scaling up of Industry 4.0 concepts.

As robots evolve from industrial machines performing repetitive tasks in isolated areas of large-scale factories to highly complex systems powered by deep neural networks, SecondHands has the ambitious goal to solve one of the greatest challenges facing the robotics field: developing collaborative robots that can safely and intelligently interact with their human counterparts in a real-world factory environment.


Graham Deacon was an invited speaker at Next Generation Robots for the Factory of the Future at the Royal Society in London on November 17, 2017

FourByThree proposes the development of a new generation of modular industrial robotic solutions that are suitable for efficient task execution in collaboration with humans in a safe way and are easy to use and program by the factory workers. FourByThree is a European project aimed to design, build and test pioneering robotic solutions able to collaborate safely and efficiently with human operators in industrial manufacturing companies. This workshop will present the robotic advancements and scientific results achieved during FourByThree project. In addition, we will present and discuss the future trends of robotics.


Tamim Asfour was an invited speaker at the IEEE-RAS Humanoids 2017 conference in Birmingham, UK on November 15, 2017

Combining Model-based and Learning-based Approaches for Humanoid Grasping at the Towards Robust Grasping and Manipulation Skills for Humanoids workshop

The ability to grasp and manipulate objects provides an essential means to interact with the environment. Recent years have seen a proliferation of research projects to use robotic manipulation in real world applications such as human robot collaboration and industrial tasks. Despite the promising progress, robotic grasping and manipulation has yet to demonstrate necessary robustness and dexterity to be fully exploited in various settings, such as in everyday life contexts, industrial environments, and when dealing with novelty and uncertainty, e.g., object shape, pose, weight, friction at contacts, and with unstructured environments.

Studies on human grasping and manipulation have shown that sensorial capabilities play a key role in the success of human manipulation, allowing a better perception of the object and the interaction with it, and revealing adaptation and control strategies, e.g., using environment and its constraints for more effective manipulation. Inspired by these findings, robotics research aiming to robustify object grasping and manipulation skills shows the importance of effective use of sensory data (visual, tactile, proprioceptive) from planning stage to task completion. Various kinds of approaches have been proposed, e.g., data-driven and empirical approaches such as learning from experience and from human demonstration, analytic approaches such as modelling physical and dynamical constraints manually, and approaches to hand designs such as under-actuated and soft hands.

In this workshop, we aim to bring together researchers and experts in key areas for grasping and manipulation such as perception, control, learning, design of hands and grippers, and studies analysing human manipulation skills. We aspire to identify recent developments in these research areas, both in theory and applications, discussing recent achievements, debating underlying assumptions, and challenges for future progress.


Prof. Tamim Asfour was an invited speaker at the Humanoid Robots for Real Applications Use workshop, IEEE-RAS Humanoids 2017 in Birmingham, UK on November 15, 2017

ARMAR-6: Humanoid Robots for Maintenance tasks in warehouse Environments 

Whereas traditional robotic systems have paved the path in various applications and are renewing the landscape of future industry and our societal culture, the main usage of humanoid robot platforms is still confined to laboratory research and development despite the noticeable advances that are made in various aspects of humanoid technology. Yet recent needs that appeared from various application fields would be ready to adopt humanoid technology shall it fulfill specifications in terms of functionalities but also reliability and safety.

The problem that we witness now is that no humanoid platform is readily available to make such a jump, and more importantly, no known robotic arms industry seems to be strongly willing to build such systems for effective business, with the exception of SoftBank Robotics for domotics and marketing.

The aim of this workshop is to gather speakers around a topic of high importance in humanoid research and development: how far are we in achieving humanoid robots or transfer humanoid technology for real-use applications? What are the ingredients that make such an endeavor possible? What are the bottlenecks in terms of research and development that could make such a perspective a viable one?

Therefore, the workshop will gather a set of the main representative actors in current projects of well-identified end-users of humanoid technology and those industries that can potentially build it. The idea is to share opinion and raise discussions not on the research perspectives only but also on the non-said practical issues that are fuzzy and not clear to researchers. For instance, what business-plans are possible for humanoid robots? How costs can be reduced despite the complexity of the structure? What inhibits well-known robotic arms providers to build and commercialize humanoid robots? What prohibits companies that have already the technology to go further with innovation plans? From the industrial viewpoints, where research and development of humanoids should focus?

Website: http://jrl-umi3218.github.io/workshop-humanoids-2017.html#content



Prof. Tamim Asfour was an invited speaker at the IEEE/RJS IROS conference in Vancouver, Canada on September 24, 2017

One of the key skills for a robot is to physically interact with the environment in order to achieve basic tasks such as pick-and-place, sorting etc. For physical interaction, object grasping and manipulation capabilities along with dexterity (e.g. to use objects/tools successfully) and high-level reasoning (e.g. to decide about how to fulfill task requirements) are crucial. Typical applications of robots have been welding, assembly, pick-and-place in industrial settings. However, traditional industrial robots perform their assignments in cages and are heavily dependent on hard automation that requires pre-specified fixtures and timeconsuming programming.

This workshop focuses on human-in-the-loop robotic manipulation that can involve different human roles, e.g., supervisory, cooperative, active or passive. This workshop proposes to gather experts in human-in-the-loop robotic manipulation, for detecting synergies in the frameworks proposed to observe and model the human contribution to the task. We would like also to identify the critical challenges still to be addressed by the community, to reach the envisioned human-robot close and fluent collaboration, across the different approaches pertaining to the workshop topic.