Loading...
5 results
Search Results
Now showing 1 - 5 of 5
- Multi-scale cortical keypoints for realtime hand tracking and gesture recognitionPublication . Farrajota, Miguel; Saleiro, Mário; Terzic, Kasim; Rodrigues, J. M. F.; du Buf, J. M. H.Human-robot interaction is an interdisciplinary research area which aims at integrating human factors, cognitive psychology and robot technology. The ultimate goal is the development of social robots. These robots are expected to work in human environments, and to understand behavior of persons through gestures and body movements. In this paper we present a biological and realtime framework for detecting and tracking hands. This framework is based on keypoints extracted from cortical V1 end-stopped cells. Detected keypoints and the cells’ responses are used to classify the junction type. By combining annotated keypoints in a hierarchical, multi-scale tree structure, moving and deformable hands can be segregated, their movements can be obtained, and they can be tracked over time. By using hand templates with keypoints at only two scales, a hand’s gestures can be recognized.
- A disparity energy model improved by line, edge and keypoint correspondencesPublication . Martins, J. C.; Farrajota, Miguel; Lam, Roberto; Rodrigues, J. M. F.; Terzic, Kasim; du Buf, J. M. H.Disparity energy models (DEMs) estimate local depth information on the basis ofVl complex cells. Our recent DEM (Martins et al, 2011 ISSPlT261-266) employs a population code. Once the population's cells have been trained with randorn-dot stereograms, it is applied at all retinotopic positions in the visual field. Despite producing good results in textured regions, the model needs to be made more precise, especially at depth transitions.
- Disparity energy model with keypoint disparity validationPublication . Farrajota, Miguel; Martins, J. C.; Rodrigues, J. M. F.; du Buf, J. M. H.A biological disparity energy model can estimate local depth information by using a population of V1 complex cells. Instead of applying an analytical model which explicitly involves cell parameters like spatial frequency, orientation, binocular phase and position difference, we developed a model which only involves the cells’ responses, such that disparity can be extracted from a population code, using only a set of previously trained cells with random-dot stereograms of uniform disparity. Despite good results in smooth regions, the model needs complementary processing, notably at depth transitions. We therefore introduce a new model to extract disparity at keypoints such as edge junctions, line endings and points with large curvature. Responses of end-stopped cells serve to detect keypoints, and those of simple cells are used to detect orientations of their underlying line and edge structures. Annotated keypoints are then used in the leftright matching process, with a hierarchical, multi-scale tree structure and a saliency map to segregate disparity. By combining both models we can (re)define depth transitions and regions where the disparity energy model is less accurate.
- Multi-scale keypoint annotation: a biological approachPublication . Farrajota, Miguel; Rodrigues, J. M. F.; du Buf, J. M. H.The primary visual cortex employs simple, complex and end-stopped cells to create a scale space of 1D singularities (lines and edges) and of 2D singularities (line and edge junctions and crossings called keypoints). In this paper we show first results of a biological model which attributes information of the local image structure to keypoints at all scales, ie junction type (L, T, +) and main line/edge orientations. Keypoint annotation in combination with coarse to fine scale processing facilitates various processes, such as image matching (stereo and optical flow), object segregation and object tracking.
- Optical flow by multi-scale annotated keypoints: A biological approachPublication . Farrajota, Miguel; Rodrigues, J. M. F.; du Buf, J. M. H.Optical flow is the pattern of apparent motion of objects in a visual scene and the relative motion, or egomotion, of the observer in the scene. In this paper we present a new cortical model for optical flow. This model is based on simple, complex and end-stopped cells. Responses of end-stopped cells serve to detect keypoints and those of simple cells are used to detect orientations of underlying structures and to classify the junction type. By combining a hierarchical, multi-scale tree structure and saliency maps, moving objects can be segregated, their movement can be obtained, and they can be tracked over time. We also show that optical flow at coarse scales suffices to determine egomotion. The model is discussed in the context of an integrated cortical architecture which includes disparity in stereo vision.