Image Motion

Introduction

The problems of tracking and registration both deal with multiple images and are important to a wide variety of applications. It can also be important to combine the processes of tracking or registration and segmentation so that information about more than position and velocity can be obtained.

In automated segmentation and tracking of objects in image sequences, the objects can be either user specified (by the operator pointing and clicking at the beginning of the sequence), or automatically specified (by custom-developed algorithms which define the objects to be tracked).

Examples of custom tracking applications are tracking cells in turbulent flow and machine control (found underneath). The tracking of user specified objects, using our generic tracking software Vedda, is illustrated in a fish tracking example found below.  Our registration algorithms are applied to 2D gels.

Custom tracking and segmentation algorithms

Cell Tracking

The images in this example illustrate a complex problem — tracking and segmenting a large number of objects in close proximity that are not moving in a consistent manner. The objects in these images are blood platelet cells stained with fluorescent dye that are pumped through a thin chamber. By tracking and segmenting all platelets we are able to get information about size, brightness, velocity and position which can be used to characterize what are obviously complex scenes.

The left hand side shows the input images while the right shows the segmentation and tracking results. Each successfully tracked object is labelled with a number, and each number is maintained as long as the object is successfully tracked.

Machine control

A second application is in control of machines for automation of complex tasks. In this example a video signal from an ultrasound sensor was processed (using a specially designed detection and tracking procedure) in order to provide control information to a robotic saw. This problem is complex for a number of reasons

1. Ultrasound images are inherently noisy.
2. Processing needed to be fast (at least 10 frames per second) to provide control signals for the saw to move at sufficient speed.
3. The objects (cattle carcasses) vary significantly so it is desirable to minimize the amount of prior (hardcoded) knowledge used by the system.

The sequence below shows the results from a single run, with the straight line indicating the position to which the saw is being guided. The system is attempting to guide the saw down a bone that is indicated by the darker region in the ultrasound image.

 

General tracking and segmentation – using Vedda

The previous examples illustrate a custom tracking and segmentation system. On some occasions it is useful to be able to track and segment in a semi-interactive fashion using our generic tracking package, Vedda. This avoids the need to customize the procedure for locating the objects initially. The user indicates objects that need to be tracked and the Vedda software then tracks and segments the objects automatically. The results are position and shape information. This has wide application as it is not strongly dependent on models describing the shape or motion of the objects being tracked.

 

Fish Tracking

The objects in these examples are larval fish. Information of the type produced by the tracking and segmentation process allows behaviour of the fish to be analysed. In this particular example the information was used to develop an understanding of the swimming ability of larval fish to improve management of the Great Barrier Reef . The shape information was used to detect feeding behaviour.

Registration

Registration is a process that aims to align two or more images, rather than identify corresponding objects between images. Registration is commonly used in medical imagery to compare different patients, or the same patient before and after treatment. It is also closely related to the sort of processing that is necessary in stereo reconstruction. In some situations the ultimate aim is to identify corresponding objects in images, but the objects themselves are not obviously similar. In these situations a registration step can make the problem more tractable.  An interesting example is registration of two dimensional electrophoresis gels.