Appendix 5 – Morphological Studies – Methods of Recording Shape for Biomimeticists

Summary

This contains chronological study of recent techniques in recording biological shapes. In this case for the purposes of reproduction. It describes several techniques for recording shape, for different intentions.

It includes detailed descriptions of the use of confocal microscopy and atomic force microscopy.

Table of Contents

1Introduction

1.1Morphological Studies

The following papers were considered in a search for the most effective way to gather the three dimensional data of a small biological structure and the substrate with which it interacts. This research built upon previous experience with a project on 3-dimensional digitising using surface modelling in 2002 in the final year course of an undergraduate degree in mechanical engineering.

2“The relative success of some methods for measuring and describing the shape of complex objects” (1998) T McLellan, J A Endler [1]

2.1Definitions

Cladistics: a system of phylogenic classification that uses certain features of organisms, called shared derived characters, to establish evolutionary relationships

Systematic biology: Biological information organised in a taxonomic or phylogenic manner.

2.2Introduction

This paper is introduced because it is an investigation into shape and methods for measuring, describing and comparing shape. The research was conducted by a geneticist and a zoologist. The tools that they employed were mathematical. The ambition of the research was taxonomic. However, when considering this information from a biomimetic approach there are reasons why systematics is a useful tool. If evolution of structures takes place for a reason then in theory studying patterns of structures is a method of understanding and perfecting the modeling and reproduction of those structures in the absence of DNA control.

For instance the Fibonacci sequence falls into the category of systematics. If a biomimeticist was contemplating reproducing some structure that demonstrates the Fibonacci properties, then in the absence of DNA controlling the development of the structure, it would be appropriate to enter the Fibonacci sequence into the controlling algorithm in order to generate this property in the artificial structure.

Similarly cladistics is a useful tool to a biomimeticist since the results of cladistic studies form a library of natural structures and their permutations. For instance the hooks of insect tarsi have different configurations in different species and there are also configurations that do not exist.

2.3Materials and Methods

Leaf specimens were chosen from tree species that exhibited distinctive shape to a trained zoologist’s eye: ten species of maple, one species of sweetgum and two species of oak.

Figure 1 – The ten species of leaf shapes used in the study from [1]

The outlines of these specimens in Figure 1 were digitized and various mathematical methods applied to study and compare their properties:

1. Fourier analysis upon the outlined profile (a well-known curve-fitting tool) and

2. Single-parameter shape descriptors:

   1. Fractal Dimension,

   2. Dissection Index,

   3. Margin Roughness and

   4. Perimeter/length ratio

The Fourier Analysis is relatively complex and described in full in the paper, so it is felt to be inappropriate to completely describe the methodology in detail here. Unlike the methods employing single-parameter shape descriptors it uses actual Cartesian co-ordinates which are processed and examined for correlations using a range of harmonics (see Figure 2 – The shape reproductions resulting from the Fourier Analysis upon the digitised profiles).

Figure 2 – The shape reproductions resulting from the Fourier Analysis upon the digitised profiles from [1]

Single-parameter Shape Descriptors are shape ratios and dimensionless:

• Fractal dimension is based on the use of fractal geometry. Fractal geometry, which “is characterized by self-similarity of structure or complexity of an image at all scales”, is used as a measure of the degree of complexity of a shape and it seems it is appropriate for leaf shapes and has been used by one of the authors in a previous study.

• The Dissection Index is the ratio of an outline’s perimeter to the square root of its area.

• The Margin Roughness measures small changes in the direction of an outline.

• The Perimeter/length ratio is similar to taking a measure on one’s hand from the base of the hand to tip of middle finger and comparing it with the length of the middle finger. For a leaf, the central lobe is used instead of a finger. (see Figure 3)

Figure 3 – The profile/length method from [1]

2.4Discussion

This paper demonstrates some relatively primitive methods of measuring the shape of a biological object, although the mathematics is complex in some cases. Only the first method using Fourier Analysis has any biomimetic significance but it is difficult to use compared to some of the other solutions that present themselves in the following papers.

2.5Further Papers of Interest

1. “The Fractal Geometry of Nature”, Mandelbrot, B. B. 1982, Freeman, New York

2. “Pattern and Form in repeated elements in the development of single leaves of Begonia dregei (Begoniaceae)”. Am. J. Bot. 80:796-804.

3“Real world modeling through high resolution digital 3d imaging of objects and structures” (2000) J Angelo Beraldin, F Blais, P Boulanger, L Cournoyer, J Domey, S F El-Hakim, G Godin, M Rioux, J Taylor [2]

3.1Introduction

The measurement of 3D objects can be divided into two broad categories. By contact with and non-contact with the object. This paper concerns itself with some of the non-contact methods of digitizing. For the purposes of measuring small biological structures the methods discussed in this paper are not suitable. However this paper does provide background to the topic of imaging and some pertinent remarks and ideas.

The main problem with imaging structures of the order of size of insect tarsi (~100m and smaller) is resolution. Even the most modern 3D colour laser scanners which are described in this paper are not equal to the task of imaging objects of 100m or less, when the element sizes of the photo-receptors on the CCD of a digital camera are of the same order of magnitude as the object.

This paper is written with the ambition of utilizing virtual reality. This is useful for the purpose of modeling the interactions of objects in a virtualized world. But this is not the end product of this research into biomimetics but merely a step in the process. Of greater importance is to find ways of utilizing the 3-D information to yield a useful identical structure instead of a mere computer generated model.

There is mention of the potential for connecting a 3D digitizer to a rapid prototyping device. Since a biomimetic approach is to copy shape as accurately as possible, the connecting a rapid prototyping device to a data collector with the necessary resolution goes some way towards illustrating the principle of converting 3D data to the reconstruction of the structure. There is more discussion on methods to do this later in this Appendix. But the most effective rapid prototyping devices cannot accurately reproduce objects in the 100m size range and smaller and other methods of manufacture must be found that can fulfill this need. This is briefly discussed in the section on self-assembly.

3.2The Scope of this Paper

This paper deals with non-contact techniques of measurement based upon the use of an energy source that is present or is projected upon the surface of an object. The returned energy is then collected by some means.

This paper does not discuss techniques using X-rays, microwaves, gamma rays and particle emissions. It deals strictly with the use of optical radiation. Ultrasound is another method of measurement which is used for instance in foetal imaging but it is not useful for precise measurement.

It does not deal with advanced forms of microscopy such as confocal, atomic force and scanning probe microscopy. These are discussed in the next sections.

It does discuss three optical systems: a BIRIS (bi-iris) system, a large field-of-view system and a colour scanner. The significant difference between the performances of the three is the error. The BIRIS system has an error of 0.1mm at a viewing range of 0.3m from the object, the large field-of-view system has an error of 0.1mm at a range of 0.5m and the colour scanner has an error of 0.01mm at a range of 0.55m from the object. Only the BIRIS and colour scanner systems were available commercially at the time of publication of the paper. The BIRIS system is clearly unsuitable for small biological structures and is not discussed further here.

3.3Colour Scanning

Figure 4 shows the product of a laser scanner harnessed to a turntable. A colour scanner can function either through rotating the object or by translating the camera about the object. Data algorithms then “sew” the image sequences together to produce the final 3D model.

Consider Figure 4. (a) shows the camera and the object setup. The white projected laser line can be seen on the object. The camera lacks a y scanner and it is optimize to operate over a depth of view of 200mm. In other words optimally the object should be no larger than 200mm in height. See Figure 4 (b).

Colour is added to the image by using a polychromatic laser. The returned collected light is then split close to the collecting lens. An image is composed of 1024 profiles and each profile is composed of 1024 co-ordinates of (x, y, z, R, G, B). Figure 4 (c) and (d) show two views of the object. Figure 4 (e) and (f) show how the colour of the object is converted into shading to give texture to the image. This is shown for the whole object in Figure 4 (g). Figure 4 (h) shows a wire mesh representation. This wire mesh is made up of polygons. If the mesh were made up of triangles it would probably be stored in .stl format and suitable for export to a rapid prototyping device and it would be possible to produce a resin model of the object.

Figure 4 – Product images using a colour scanner and turntable from [2]

3.4Discussion

This paper concentrates on virtual reality as an end goal, not the acquisition of information for biological or biomimetic research. It is useful to have some discussion of alternatives in the field of digital imaging, however, before moving on to techniques more suitable to biomimetic research.

3.5Papers of Further Interest

There are a large number of papers in the references of this paper which mostly deal with computer vision and virtual reality. Therefore they are not listed and the discussion of this paper is drawn to a close here.

4“An assessment of data requirements and data transfer formats for layered manufacture” (1998) A L Marsan, V Kumar, D Dutta, M J Pratt [3]

4.1Introduction

Layered manufacture, presented here in the context of morphological studies, seems at first inappropriate. However, layered manufacture is a form of reverse morphological study. It is forward engineering instead of reverse engineering.

This standard, an American standard for layered manufacture, is an in depth description of the various stages of the Layered Manufacture process. The following has been extracted as being a relevant but not complete description of the contents. The topics describe both generalized and specific descriptions of the current state of technology at that time (1998). Such is the nature of technological development that different research paths are followed in order to solve different problems and this standard is a unifying concept.

It must be born in mind that this research paper is always oriented towards working with hook-shaped structures which are a relatively simple shape. The only complexity to the shape is the presence of an undercut in a complete hook but this can be overcome by considering the hook to be a curved object that lies upon its longitudinal axis, i.e. by laying the hook upon its side, particularly in the imaging phase which is described in Section 5.

4.2Layered Manufacture

In product design, layered manufacture forms part of the product design process and it is used in conjunction with specialized software and hardware tools throughout the product development cycle. Detail design includes analysis and geometric modeling.

During the layered manufacture process planning phase for a part, one of the processes consists of slicing the geometric model of the part into 2-D slices and generating tool paths. This is in effect the reverse of the 3-D reconstruction that occurs in the reconstruction of the 3-D confocal imaging process where an image of a biological shape is scanned in slices and presented in the data form of a stack of image slices (see Section 5 later in this appendix for confocal microscopy).

Data describing the boundary of an object is most commonly transferred to the process planning system in the form of a STL file. The STL file format was originally developed for stereolithography.

.STL represents the boundary of the object to be manufactured by a mesh of triangular facets.

A layered manufacturing process requires the description of the boundary of the object to be manufactured. This is not the only information required, since manufacturing information includes other information such as material, tolerances, surface finishes etc.

This information about the boundary of an object can be supplied in a number of forms:

1. Surface Models

2. Solid Models

3. Point Data

4. Mathematical Data

5. Image Data

6. Slice Data

Items 1, 2 and 4 are associated with forward engineering, designing using software designed for that purpose.

The image files created by the confocal microscope bear the greatest resemblance to items 3, 5 and 6 above.

Item 3, Point Data, is mostly used in engineering for reproducing a existing part. The part is digitized using a touch probe or a laser scanner. But, according to the standard, this information is usually insufficient for layered manufacture and requires further processing.

Item 5, Image Data, provides information not just about the exterior of the part but also interior information such as material density.

Item 6, Slice Data, provides information in the form of contours instead of pixels or voxels. This information may be suitable for direct application to layered manufacture if the slice thicknesses and shape orientation fulfill LM requirement. If not then the information must be converted to a surface model which can then be orientated and sliced during the LM process.

5“Confocal imaging, visualisation and 3-D surface measurement of small mammalian teeth” (2001) A R Evans, I S Harper, G D Sanson [4]

5.1Introduction

This paper uses confocal microscopy to image the surfaces of bat teeth and produce 3 dimensional images of the interactions o upper and lower jaws in order to study wear patterns.

It demonstrates that for biological structures there is no single method of obtaining a model of any structure. In this case the teeth are of the order of 2mm x 2 mm and of tooth enamel which is opaque to laser light and non-fluorescent. A method is required to obtain the 3D information of the teeth and this is described in the paper.

It was demonstrated using a calibration object that it was possible to obtain images accurate to within 4% of the original.

5.2Materials and Method

The paper describes techniques of incorporating a fluorescent dye called eosin into the cats of a tooth. Confocal microscopy allows the reconstruction of surface topology either by reflection using gold sputter-coated replicas or the teeth themselves, or by fluorescent imaging of a cast of the teeth.

It is stated that using the teeth themselves or gold-sputtered replicas is not suitable because of excessive reflection when the actual teeth are used and because the construction of the microscope without internal anti-reflection coatings makes them inadequate for imaging in reflection mode.

Further, staining of the surface of a cast with eosin gave the strongest signal.

In order to study the technique prior to imaging the teeth themselves a glass specimen was prepared of precise dimensions. This was used to test and calibrate the technique and to arrive at the conclusion that accuracy of the scan was found to be within 4% of true dimension.

The teeth used in this experiment were substantially larger than the insect tarsi for which this imaging method was being contemplated. There is extensive analysis of the optimum performance in the use of the microscope (a Leica TCS_NT single phase confocal microscope).

Imaging was conducted using dry objectives through rhodamine optics (excitation 568nm) with fields of view of 2 x 2 mm and 1 x 1 mm respectively. The pinhole was set at 1.00 optical unit.

Optical slices were taken through the object in the x, y, plane where each slice was a square (256 x 256) pixel 8-bit image at medium scanning speed. Slices were taken at the same distance as the interval between pixels to make cubic voxels (volume pixels).

“Powerscan” software was then used to generate an 8-bit image of the stack of slices where pixel intensity represented height, by detecting the z-height with the greatest fluorescence of each x, y point. Other software used was “Cosmo Player 2.1” which when calibrated in the x, y and z planes can generate a proportioned representation of the tooth on the screen.

In the study of surface noise it was found that with lenses of a small depth of field (with a high numerical aperture) the surface noise was minimized.

5.3Papers of Further Interest

1. “Capturing data from three-dimensional surfaces using fuzzy landmarks” Valeri C J, Cole T M, Lele S, Richtsmeier J T (1998) American Journal of physical Anthropology 107, 113 – 124

2. “Applications of confocal microscopy to functional morphology: the effect of diet and wear on tooth sharpness and function in two microchiropterans Chalinolobus gouldii and C. morio.” Proceeding of the 9^th Australian Bat Conference, Tocal, Australia, 25-28 April 2000. Bat Res. News, 41, 44

3. “Measurement accuracy in confocal microscopy” Delorme R, Benchaib M, Bryon PA, Souchier C (1998), Journal of Microscopy 192, 151-162

6“Scanning probe microscopy imaging and characterization of biological structures from biomolecules to living cells” (1994) V B Pizziconi, D L Page, C T Connolly, P A Diamond [5]

6.1Introduction

This paper discusses scanning probe microscopy (SPM) and specifically two methods of SPM, scanning tunneling microscopy (STM) and atomic force microscopy (AFM). It was written in 1994 and such is the rate of technological advance that it can be assumed that much progress has been made since it was written.

However, since this is not research into microscopy but a survey for techniques suitable for biomimetics, the discussions of the principles are relevant.

In this case the immediate application of SPM to the problem at hand, that of biological attachment systems, is to reproduce the substrate morphologies with which a hook interacts. Certainly it is possible that confocal microscopy could be suitable for this application. However, a substrate is normally considered to be a 2-dimensional surface with irregularities (see Appendix 3 on the plant epidermis). This makes it more suitable for microscopy of a different type, that does not require the transmission of light but instead that is opaque and reflective to certain wavelengths of light.

6.2Description of Scanning Probe Microscopy

It seems appropriate to quote directly from the text:

“As a class, SPM generically represents any method that precisely controls the movement of either a probe or sample in three dimensions by piezoelectric elements as a means to obtain probe-to-surface distances that result in physical interactions that are transducible to high resolution surface topography. It presently includes scanning tunneling microscopy (STM) and atomic force microscopy (AFM) methods which are commercially available. Other SPM techniques currently under development include scanning ion, magnetic force, photon scanning tunneling, and thermal profiler microscopes. The extent and diversity of SPM development and applications are impressive for such a relatively short time period since SPM’s inception and these are summarized elsewhere.

The scanning elements of STM and AFM, depicted in Figure 5 and Figure 6 respectively later in this section, distinguish the basic operating characteristics between these SPM-related but distinctly different microscopy methods. Briefly, STM exploits electron tunneling phenomena, which is favoured to occur when a conductive probe is placed in very close proximity to a conductive surface separated by only a thin insulating gap whereas AFM is based on attractive/repulsive forces occurring between a cantilever mounted probe and a sample and is not restricted to conductive probe/sample systems.”

Figure 5 – Schematic of scanning tunnelling microscopy operation from [5]

Figure 6 – Schematic of Atomic force Microscopy Operation from [5]

“A particularly attractive generic feature of these SPM systems is their ability to obtain both two- and three-dimensional topographical features of material surfaces at extremely high resolution in either vacuum, air, or liquid environments. It is this latter ability of SPM systems, i. e., the imaging if biological structure directly in a liquid environment, that is the most distinguishing from other high resolution microscopies used for imaging biological materials.”

Overleaf is a table of structures, molecules and organelles that had been imaged using SPM by 1994.

Figure 7 – Imaging using SPM (a) from [5]

Figure 8 – Imaging using SPM (b) from [5]

6.3Discussion

Although as yet, no experiments have been conducted using SPM in biological attachment mechanisms, it is clear from the research that it is suitable for experimental purposes, particularly for creating models of hook/surface interactions as discussed by Gorb in Appendix 4. By converting the data to .stl format it should be possible to produce scale resin models of both hook and substrate and study their interactions.

“Living cell experiments indicate that STM can provide unique insight into several areas of importance in molecular cell biology and cellular bioengineering, including cell-cell recognition and adhesion, cell-substrate adhesion, cell signalling, and cell growth and differentiation, among others.

The text mentions that SPM has been used to study hydroxyapatite, a bone substitute that is self-assembled onto biological implants such as hip replacements. It is anticipated to investigate the feasibility of using this form of microscopy for leaf surfaces and converting the file format to .stl as part of future work.

7“3D modelling of biological systems for biomimetics” (2004) S Zhang, K Hapeshi, A K Bhattacharya [6]

7.1Introduction

This paper is a discussion of methods for 3-D imaging and reconstruction for biomimeticists. It highlights some of the problems that have been experienced using different forms of 3D reconstruction and suggests that there is no “bible” to dictate which of the types of 3D analysis is most suitable.

7.2Reverse engineering method of creating 3D models

This is a procedure described in the paper where the reverse engineering approach is differentiated from the normal engineering method of direct generation on a computer using software such as Solid Works.

The reverse engineering process is described as follows;

1. Data Acquisition

2. Data Processing

3. Segmentation

4. Surface Model Generation

5. Solid Model Creation

Bearing in mind that this paper is a generalized approach to 3D imaging there is much in the content that is not applicable to a study of biological attachment mechanisms and hooks in particular. This is because of the relatively simple shape of the hook and the possibility of using some very easy techniques such as confocal microscopy to yield a stack of images that are already registered in position with respect to each other.

This eliminates the need for step three, the segmentation process, when an irregular biological shape is separated into its component volumes.

8References

1. “The relative success of some methods for measuring and describing the shape of complex objects”, T McLellan, J A Endler, Journal of Systematic Biology 47(2): 264-281, 1998

2. “Real world modelling through high resolution digital 3D imaging of objects and structures” (2000) J Angelo Beraldin, F Blais, P Boulanger, L Cournoyer, J Domey, S F El-Hakim, G Godin, M Rioux, J Taylor, ISPRS Journal of Photogrammetry and Remote Sensing 55 (2000) 230-250

3. “An assessment of data requirements and data transfer formats for layered manufacture” (1998) A L Marsan, V Kumar, D Dutta, M J Pratt, US Dept of Commerce, NISTIR 6216

4. “Confocal imaging, visualisation and 3-D surface measurement of small mammalian teeth” (2001) A R Evans, I S Harper, G D Sanson, Journal of Microscopy, Vol 204, Pt 2, November 2001, pp 108-119

5. “Scanning probe microscopy imaging and characterization of biological structures from biomolecules to living cells” V B Pizziconi, D L Page, C T Connolly, P A Diamond from ”Atomic force microscopy/scanning tunneling microscopy” (1994) M T Bray, M L Lightbody, edited by S H Cohen, Plenum Press, New York, ISBN 0-306-44890-4

6. “3D modelling of biological systems for biomimetics” (2004) S Zhang, K Hapeshi, A K Bhattacharya, Journal of Bionics Engineering, Vol 1, No 1, ISSN 1672-6529