Untitled

Funder

Organizational Unit

Publications

The SmartVision Navigation Prototype for Blind Users

Publication . du Buf, J. M. H.; Barroso, João; Rodrigues, J. M. F.; Paredes, Hugo; Farrajota, Miguel; Fernandes, Hugo; José, João; Teixeira, Victor; Saleiro, Mário

The goal of the Portuguese project "SmartVision: active vision for the blind" is to develop a small, portable and cheap yet intelligent and reliable system for assisting the blind and visually impaired while navigating autonomously, both in- and outdoor. In this article we present an overview of the prototype, design issues, and its different modules which integrate GPS and Wi-Fi localisation with a GIS, passive RFID tags, and computer vision. The prototype addresses global navigation for going to some destiny, by following known landmarks stored in the GIS in combination with path optimisation, and local navigation with path and obstacle detection just beyond the reach of the white cane. The system does not replace the white cane but complements it, in order to alert the user to looming hazards. In addition, computer vision is used to identify objects on shelves, for example in a pantry or refrigerator. The user-friendly interface consists of a four-button hand-held box, a vibration actuator in the handle of the white cane, and speech synthesis. In the near future, passive RFID tags will be complemented by active tags for marking navigation landmarks, and speech recognition may complement or substitute the vibration actuator.

2011Journal article

Open access

Face and object recognition by 3D cortical representations

Publication . Martins, Jaime Afonso do Nascimento Carvalho; du Buf, J.M.H.; Rodrigues, J.M.F.

This thesis presents a novel integrated cortical architecture with significant emphasis on low-level attentional mechanisms—based on retinal nonstandard cells and pathways—that can group non-attentional, bottom-up features present in V1/V2 into “proto-object” shapes. These shapes are extracted at first using combinations of specific cell types for detecting corners, bars/edges and curves which work extremely well for geometrically shaped objects. Later, in the parietal pathway (probably in LIP), arbitrary shapes can be extracted from population codes of V2 (or even dorsal V3) oriented cells that encode the outlines of objects as “proto-objects”. Object shapes obtained at both cortical levels play an important role in bottom-up local object gist vision, which tries to understand scene context in less than 70 ms and is thought to use both global and local scene features. Edge conspicuity maps are able to detect borders/edges of objects and attribute them a weight based on their perceptual salience, using readily available retinal ganglion cell colour-opponency coding. Conspicuity maps are fundamental in building posterior saliency maps—important for both bottom-up attention schemes and also for Focus-of-Attention mechanisms that control eye gaze and object recognition. Disparity maps are also a main focus of this thesis. They are built upon binocular simple and complex cells in quadrature, using a Disparity-Enery Model. These maps are fundamental for perception of distance within a scene and close/far object relationships in doing foreground to background segregation. The role of cortical disparity in 3D facial recognition was also explored when processing faces with very different facial expressions (even extreme ones), yielding state-of-the-art results when compared to other, non-biological, computer vision algorithms.

2013Doctoral thesis

Open access

Correspondence of three-dimensional objects

Publication . Lam, Roberto; du Buf, J. M. H.

First many thanks go to Prof. Hans du Buf, for his supervision based on his experience, for providing a stimulating and cheerful research environment in his laboratory, for letting me participate in the projects that produced results for papers, thus made me more aware of the state of the art in Computer Vision, especially in the area of 3D recognition. Also for his encouraging support and his way to always nd time for discussions, and last but not the least for the cooking recipes... Many thanks go also to my laboratory fellows, to Jo~ao Rodrigues, who invited me to participate in FCT and QREN projects, Jaime Carvalho Martins and Miguel Farrajota, for discussing scienti c and technical problems, but also almost all problems in the world. To all persons, that worked in, or visited the Vision Laboratory, especially those with whom I have worked with, almost on a daily basis. A special thanks to the Instituto Superior de Engenharia at UAlg and my colleagues at the Department of Electrical Engineering, for allowing me to suspend lectures in order to be present at conferences. To my family, my wife and my kids.

2017-06-23Doctoral thesis

Open access

Cognitive robotics: a new approach to simultaneous localisation and mapping

Publication . Saleiro, Mário; Rodrigues, J. M. F.; du Buf, J. M. H.

Most simultaneous localisation and mapping (SLAM) solutions were developed for navigation of non-cognitive robots. By using a variety of sensors, the distances to walls and other objects are determined, which are then used to generate a map of the environment and to update the robot’s position. When developing a cognitive robot, such a solution is not appropriate since it requires accurate sensors and precise odometry, also lacking fundamental features of cognition such as time and memory. In this paper we present a SLAM solution in which such features are taken into account and integrated. Moreover, this method does not require precise odometry nor accurate ranging sensors.

2010Conference object

Open access

Cortical 3D Face Recognition Framework

Publication . Rodrigues, J. M. F.; Lam, Roberto; du Buf, J. M. H.

Empirical studies concerning face recognition suggest that faces may be stored in memory by a few canonical representations. In cortical area V1 exist double-opponent colour blobs, also simple, complex and end-stopped cells which provide input for a multiscale line/edge representation, keypoints for dynamic routing and saliency maps for Focus-of-Attention. All these combined allow us to segregate faces. Events of different facial views are stored in memory and combined in order to identify the view and recognise the face including facial expression. In this paper we show that with five 2D views and their cortical representations it is possible to determine the left-right and frontal-lateral-profile views and to achieve view-invariant recognition of 3D faces.

2011Conference object

Open access