Opal Grading

Figure 1: Typical opal images – (Top Left) black opal, (Top Right) white opal, (Bottom Left) boulder opal, (Bottom Right) crystal opal

Introduction:

Current opal assessment and evaluation practices are highly subjective as they are based on a combination of human observed factors, such as scoring the flashes of colour as the opal is moved, the body tone, colour, brightness and pattern. The fact that Australia contributes 95% of the world’s opals is both a blessing and a curse. Many participants in the opal supply chain do not have the training to assess the value of opals being supplied (leading to lack of confidence to trade and difficult negotiations between buyer and seller), nor can they describe a particular type or quality of opal in sufficiently concise and objective terms for the suppliers to be able to provide the appropriate opals.

Most gemstone assessment is made using a magnification 10x loupe. The gemstone is observed by holding it in a pair of gemstone tweezers and turning it (using pitch, roll and yaw) to observe light interaction with the gemstone and external and internal characteristics. There exist a number of key characteristics on which an opal can be graded for its value, such as: Colour (hue) and area of flash, Brightness, Body tone, Pattern, Shape and Other characteristics.

In terms of relative importance to the overall gemmological value of an opal, colour (hue, brightness, saturation and body tone) has over 80% weighting, with the remaining characteristics being of much less significant weighting. In terms of seeking an objective assessment by a human observer, colour (hue, brightness and saturation and body tone) appears to be the most difficult, followed by pattern, which seems easier to assess and quantify manually, of which there are many types.

Colours in an opal are difficult for a human observer to assess because of the following:

The contribution of each colour changes with viewing angle, that is, with pitch, roll and yaw. All orientations are to be integrated in a colour estimation, which is difficult due to the limited perception and subjective memory of humans and their eyesight variables.
Out in the field, it is difficult to accurately assign a shade according to a colour reference chart because the conditions for viewing are highly variable.

The Gemmological Digital Analyser (GDA) is a vision system developed by CSIRO and Applied Robotics under contract to the Opal Produces of Australia Limited (OPAL) for the automated grading of opals. The following describes the hardware and software developed for automated assessment of the flash colour and bodytone of opals.

Image Capture:

A camera captures a series of images at multiple angles in order to quantify the flash over the full range of viewing angles. The light source is positioned as close as possible to the camera so that the illumination angle and viewing angle are nearly coincident. This is to replicate the lighting conditions used for human opal grading. Two image capture geometries are possible: tilting the rotation stage holding the stone while holding the camera and light source fixed; or moving the camera and light source while holding the tilt angle of the rotation stage fixed. We have chosen the fixed camera option. We have established that a sampling frequency of 10 degrees is necessary in both rotation and tilt angles to ensure that we do not miss any flash regions. The following is the image capture procedure:

The camera is positioned directly above the rotation stage containing the opal and initially the stage is positioned at a tilt angle of 0 degrees i.e. so that the angle between the plane of the stage and the camera is 90 degrees.
With back lighting, 1 image is captured to determine if the stone is translucent (i.e. crystal) and also to determine the region of interest (ROI) within the image that contains the stone.
With forward lighting, 36 Images are captured at 10 degree intervals for 0 to 360 degrees of stage rotation. This series of images is repeated for 10 degree steps of tilt angle of the stage until a side view of the stone is achieved.

Figure 2: Sample preparation technique

The relative camera and stone positions are illustrated by Figure 2 although in reality it is the stage which tilts and rotates while the camera remains fixed.

Image Calibration:

The calibration of each image to international standards entails the following steps:

Extended exposure – multiple exposure images are combined to extend the dynamic range of the low-noise (Peltier cooled) camera; this is necessary to cover the very wide dynamic range from very dark areas of bodytone to flash regions that are typically orders of magnitude brighter.
Lighting correction – to correct for lighting non-uniformities across the field of view of the camera.
Colour calibration – to convert from device specific RGB to device independent XYZ values then to device-independent sRGB with a non-linear transfer function (gamma curve).

The device specific characteristics of the system are determined by imaging standard Kodak white and grey cards to measure lighting uniformity within the field of view and by imaging a Gretag-Macbeth colour card to measure the colour response of the lights and camera. This System Calibration procedure is conducted once per week, or once every 100 stones captured, whichever is soonest, and also following the replacement of any major component within the system. This ensures that the device specific characteristics are updated whenever the system changes. This is particularly necessary because the brightness and colour temperature of the halogen lights drift over time.

The non-linear transfer function (gamma curve) used in the device-independent sRGB images closely matches that of the human visual system. If sRGB images of opals are viewed on sRGB calibrated monitors they will closely match the actual opal appearance if viewed under natural daylight (D65 lighting conditions). A detailed description of the image calibration procedure is given here.

Gammad sRGB to non-linear Look-Up Tabled sRGBlut

The sRGB standard is designed for display of images of diffuse reflecting objects such as the bodytone of the opal. Because the flash is specular reflection from the internal structure of the opal, it can be orders of magnitude brighter. No single image display will be able to accurately represent the colour of both the bodytone and the flash. For this reason, we have used a Look-Up Table (LUT) to compress the dynamic range of the flash so that a realistic view of the bodytone can co-exist with an approximate representation of the flash. This LUT is designed to be linear in the brightness range of the bodytone, 0-Bflash (where Bflash ~500) and to compress flash brightness >Bflash as shown in Figure 3. Applying this LUT to IsRGB gives the image IsRGBlut, as shown in Figure 4. Note that IsRGB shows the true colour of the bright flashes but the bodytone is quite dark. IsRGBlut however gives a realistic representation of bodytone colour but the bright orange flash regions appear more yellow because of saturation. Note that this IsRGBlut image is not for measurement but for display purposes only.

Figure 3 LUT is a transfer function between input brightness and output brightness.

Figure 4  Comparison of standardised image IsRGB and LUT-compressed image IsRGBlut.

(Left) IsRGB which shows the true colour of the bright flashes but the bodytone is relatively dark. (Right) IsRGBlut which gives a true representation of bodytone colour but the bright orange flash regions appear yellow because of saturation. Horizontal transects through the centre of both images are shown below.

Colour transformation from sRGB to sHSB

The RGB representation of colour is commonly used in image capture and display devices, but it is not designed for describing human perceptions of colour. For this we transform to an alternative representation of colour called HSB (or HSV) as shown in Figure 5. HSB stands for Hue, Saturation and Brightness (also known as Value). Hue is a measure of the wavelength of a colour and is given as an angle between 0 and 360 degrees. Saturation is a measure of the purity of colour or the amount of white added. A pure colour will have 100% saturation. For decreasing values of saturation, the pure colour is increasingly diluted with white. The Brightness (or Value) is a measure of the intensity of the colour. The brightest colour will have 100% brightness. For decreasing values of brightness, the pure colour is increasingly diluted with black. In our extended exposure images, our brightest colour Bmax (100%) has been scaled to 3000.

H hue is a measure of the wavelength of a colour and is given as an angle between 0 and 360 degrees.S saturation is a measure of the purity of colour or the amount of white added. A pure colour will have 100% saturation.B/V brightness/value is a measure of the intensity of the colour. The brightest colour will have 100% brightness.

Figure 5  HSB or HSV representation of colour

Because HSB is a simple transformation of the RGB values in an image, it is defined relative to the standards of the RGB values. Transforming sRGB values will give standardised sHSB values, relative to the D65 white point.

Segmentation:

Before any measurement can take place, the image must be segmented, i.e. the glint on the surface of the stone must be located and the stone must be separated from the background. The complete segmentation method cannot be described in detail here; we will just show a segmentation result on Figure 6:


(a)	(b)	(c)

Figure 6: A typical segmentation – (a) raw image, (b) segmented glint, (c) segmented stone

3: Flash Histogram Measurements:

Unlike bodytone which is view-angle independent, the flash in an opal changes from each viewing angle. To ensure that we do not miss any flash regions, we must capture and analyse 324 images to cover the full range of viewing angles. This is challenging to measure and even more so to display in a way which conveys a summary of these measurements.

3D histogram of stone HSB values

For each image, we have identified a stone mask which excludes the background and glint regions. We do not attempt to exclude non-opal regions or bodytone regions because these are only established at one tilt angle. We create a summary of the stone colour characteristics by taking a histogram of sHSB values present within the mask. Each histogram is a 3D array of the counts of pixels falling within bins of Hue, Saturation and Brightness value. There are 30 Hue bins linearly spaced in the range of 0 to 360 degrees. Saturation has 10 bins between Sflash (~40%) and 100%. Saturation is constrained to lie above the Sflash threshold to exclude any faint glint that is not detected by our glint detectors. This will not exclude any flash because it is highly saturated. Note however that some non-flash regions may be excluded by the Sflash threshold. Brightness has 30 bins piece-wise linearly spaced to give 10 bins in the range containing bodytone (0-Bflash) and 20 bins in the range containing strong flash (Bflash – Bmax).

The pixel count in each bin is divided by the number of pixels in the stone (including glint regions), so the bin value gives the proportion of the stone having the HSB values of that bin.

This is a very compact summary of the colour information. For example, the opal image may be 800×800 pixels. This requires 640,000 HSB values to store and display the colour information. By discarding the spatial context, the 3D histogram requires only 9,000 bins (30 H bins x 10 S bins x 30 B bins) to store this information. Also, because we have discarded the spatial context, we can add the histograms from multiple views to get the average proportions of the stone having specific HSB values.

The summary 3D stone histogram contains bin counts for both flash and bodytone regions of the stone. We can separate these out by using the knowledge that strong flash is both bright B>Bflash (500) and quite highly saturated S>Sflash (40%). [Note: these flash Brightness thresholds may be lowered if the Brightness values of the bodytone and non-opal regions are known.] The result is a 3D histogram of flash HSB values.

Summary histograms of flash H & B and H & S values

The 3D flash histogram can be stored but it is difficult to display for easy human interpretation. Consequently, we first combine all the Saturation bins and create a 2D summary histogram of Hue and Brightness values. We also combine all the Brightness bins and create a 2D summary histogram of Hue and Saturation values.

Summary H&B histograms and H&S histograms for two stones, Golden Grace and Flatspot, are shown in Figure 7.

Figure 7 Summary Hue & Brightness histograms and Hue & Saturations histogram for two stones.

Top row – Images of (Left) Golden Grace and (Right) Flatspot taken at 80deg tilt angle.
Middle row – Hue & Brightness histogram for each stone.
Bottom row – Hue & Saturation histogram for each stone.

The H&B histogram is to be interpreted as follows: Hue is plotted on the x-axis; the height of each histogram bar is the area proportion of that Hue; within each bar, gradations of brightness are used to display the proportions of the area belonging to the various Brightness bins for that Hue. Similarly, the H&S histogram is to be interpreted as follows: Hue is plotted on the x-axis; the height of each histogram bar is the area proportion of that Hue; within each bar, gradations of saturation are used to display the proportions of the area belonging to the various Saturation bins for that Hue. Note that these gradations are not as informative because flash does not tend to vary in saturation very much.

Note that according to the summary H&B histograms in Figure 7, the maximum area proportion of a single Hue in Golden Grace is only about twice that of Flatspot. However, when we examine the images (taken at 80 degree tilt angle), it is obvious where the Flatspot stone gets its name. There is a flat spot in its flash when viewed from above. This information is not at all evident in the summary histogram. For this reason, we have produced 9 additional H&B histograms to summarise this directional information.

Directional histograms of flash H & B values

The directional H&B histograms for Golden Grace and Flatspot are shown in Figure 8. Table 1 defines the range of viewing angles that have been combined for each of the histograms.

Figure 8  Directional Hue & Brightness histograms for two stones.

(Left) Golden Grace and (Right) Flatspot.

The directional histogram for Flatspot clearly shows that there is very little flash when viewed from above but it flashes green strongly from the Top Left (TL) direction. By contrast, the directional histogram for Golden Grace shows that it displays the largest area of flash and is also most colourful (flashing orange, yellow and green) when viewed from above. This directional information will be important for buyers when choosing an opal for a setting which has specific directionality constraints, such as a pendant or brooch, rather than for a ring which can be easily viewed from many directions.

Above	(tilt>=50º)
BC – bottom centre	(tilt<50º) & ((rot>337.5º) \|\| (rot<=22.5º))
BR – bottom right	(tilt<50º) & ((rot>22.5º) & (rot<=67.5º))
CR – centre right	(tilt<50º) & ((rot>67.5º) & (rot<=112.5º))
TR – top right	(tilt<50º) & ((rot>112.5º) & (rot<=157.5º))
TC – top centre	(tilt<50º) & ((rot>157.5º) & (rot<=202.5º))
TL – top left	(tilt<50º) & ((rot>202.5º) & (rot<=247.5º))
CL – centre left	(tilt<50º) & ((rot>247.5º) & (rot<=292.5º))
BL – bottom left	(tilt<50º) & ((rot>292.5º) & (rot<=337.5º))

Table 1 Definition of viewing angle ranges for directional histograms.

where tilt is the stage tilt angle and rot is the stage rotation angle.

Experimental Validation of Colour Measurement

Two aspects of the GDAs colour measurement we’re experimentally validated. The first aspect concerns the repeatability of the measurements. Having repeatable hue measurements is critical to the objectivity of the GDA. The major source of variability, in terms of repeatability, will be the manual placement of the opal on the stage. If an opal is processed by the GDA, removed and then randomly placed back on the stage, we cannot guarantee that the images produced will be identical for each process. This is because the viewing angles are being sampled in increments of 10 degrees in rotation and tilt.The output colour histograms were compared both visually and using a statistical similarity measures. The positive results of these comparisons showed that the GDA produced highly repeatable measurements.

The second validation aspect is the consistency of the GDA output with that of a human observer. To experimentally validate that this is being achieved, GDA results were compared to human assessments. This comparison was done over a number of marker opals, which were representative of the four opal types, black, boulder, opal and white. Each of these opals were manually assessed by six human graders and histograms of colour produced. A mean histogram of the manual measurements was in turn computed. A statistical measure was used to measure the difference between GDA produced histograms and the mean histogram of each opal. Similar distances were computed between the manually generated histograms and the mean histogram. Figure 9 shows a plot of these distance values. Values of D close to 1 represent a large distance between histograms. Values near zero indicate high similarity. It can be seen that the GDA produced measurements are no further away from the mean the most manual measurements. This indicates that the GDA is producing results that are consistent with manually generated data.

Figure 9: Plot of difference between mean, GDA and manually generated histograms

4: Bodytone Measurement:

Bodytone segmentation of forward lit images

In order to measure the bodytone of the stone, we must first find the viewing angle at which the bodytone is most readily visible. We avoid using the 90º stage tilt angle because this geometry often produces a large area of surface glint. The lower the stage tilt angle, the less of the stone surface is visible. So an 80º stage tilt angle is optimal. From the flash histograms, we can determine which rotation angle shows the smallest area of strong flash. This rotation angle, rotMinFlash, should have the largest area of bodytone.

Previously we have segmented the stone and excluded the glint regions and, in the case of crystal opals, excluded the internal opaque regions since these will interfere with the measurement of the bodytone of the crystal. Now we need to determine the regions within the stone mask which display bodytone before we can measure the colour characteristics of that bodytone. So first we remove the strong flash regions from the stone mask by excluding all pixels with Brightness > Bflash. We will be left with non-flash regions within the stone.

If the stone only has opal in its face, then the bodytone regions will be those where the flash is completely turned off i.e. the darkest part of the non-flash regions. Unfortunately this simple definition of bodytone region is not valid in all cases. If the stone has either potch (opal without the crystal structure which causes flash) or boulder (the rock in which the opal is embedded) present in the face, then the darkest part of the stone may well belong to these non-opal regions rather than to the bodytone of the opal present in the face.

To allow for the possibility of mixed-face opals, we search for 2 colour classes in the non-flash regions of the stone. We display the average colour of the darkest 20% of both classes and the regions they occupy within the stone and allow the operator to select the class containing the bodytone. If the stone only has coloured opal in the face, then these two classes will typically be bodytone and dark flash. If the stone has coloured opal and potch in the face, then the two classes will typically be bodytone and potch. If the stone has opal and boulder in the face, then the two classes will typically be bodytone and boulder.

Bodytone measurement

Having determined the bodytone region in the image, we take the mean or average sRGB values within this region, convert to HSB values and assign to one of the following 3D histogram bins. Just as for flash, there are 30 Hue bins linearly spaced in the range of 0 to 360 degrees, labelled HN1 to HN30. Saturation has 10 bins between 0% and 100%, labelled SN10 (white) to SN1 (pure colour). Brightness has 10 bins linearly spaced in the range containing bodytone (0-Bflash), labelled BN1 (black) to BN10 (white). We then report the H, S and B label of the bodytone.

5. Play of Colour measurement

The histograms described in Section3 are a summary of the areas of flash displayed at various viewing angles. The directional histograms are a summary of whether the flash changes in area, hue, saturation or brightness with viewing angle. These histograms do not take into account the spatial location of the flash. If the flash changes position between viewing angle changes, but retains the same area and colour, there will be no change in the histograms at these angles. This phenomenon of flash of a specific colour changing position, or a specific region of flash changing colour, as the viewing angle is changed, is termed Play of Colour. To measure this attribute, we need to compare images taken with different viewing angles.

Geometric distortion of the stone will be present if we attempt to compare images from different stage tilt angles. Therefore it is preferable to compare images taken at different stage rotation angles, or different lighting angles, for a given stage tilt angle. Because we are after an overview of the play of colour, a sampling frequency of 20 degrees is considered adequate in both rotation and tilt angle. Because Play of Colour is generally taken to be a characteristic of the above view of the stone (i.e. 50º, 60º, 70º, 80º and 90º stage tilt angles) and we wish to avoid the 90º tilt angle because it produces a large area of surface glint areas, we sample images at tilt angles of 60 and 80 degrees.

Images taken at different stage rotation angles must be rotated back in software in order to align them before they can be compared. This step is not necessary if different lighting angles are used. However, if different lighting angles are used, then each separate light source must have its own set of colour calibration files. The latter option of a secondary forward lighting source has been implemented in the GDA.

A Play of Colour measure must detect both presentations of play of colour change of colour or change of position. There are several ways of doing this. One simple method is to take the average of the absolute difference of the two co-registered views for the series of rotation and tilt angles. Example difference images are shown in Figure 10 for two stones, one with low play of colour and one with high play of colour. At a single rotation angle, the difference score is defined as the sum of the average absolute difference within the stone mask (but excluding the glint masks) of the two co-registered views for the Red, Green and Blue bands of the sRGB images.

Figure 10: Two lighting angle images and their difference image for (top) a stone showing low Play of Colour and (bottom) a stone showing high Play of Colour.

If a flash region changes colour between the two views, then the difference score will detect it. Larger colour changes will give a higher difference score, as will larger areas of colour change. If a specific flash region moves position but not colour, then the difference score will also detect this. The score can be scaled to have a range of 0 to ~100 by dividing by the difference score of the marker stone with the greatest play of colour and multiplying by 100.