Laparoscopy is a surgery technique with minimum invading:
an optic fiber (with a lighting feature) displays on a monitor
an image of the active area (with a by 3 zoom),
and adapted tools are introduced by pipes having a similar diameter.
This technique knows a high development because it has low invading,
thus few sequels, few complications, and also more precision
thanks to the zoom factor.
On the other hand, evenmore than others surgery techniques,
it needs a deep formation of praticians, done on corpses and animals.
Further than ethic and cost problems, this makes difficult the
training on precise pathologies.
There is so an intense demand for simulators.
The AISIM Incitative Action of INRIA
lies in this frame, and aims at obtaining the basis of
a laparoscopic simulator within 2 years, focused for the moment on
the liver surgery.
iMAGIS takes part on the deformation simulation
(in real-time for display + force feedback)
of the organ due to the tools action
(Gilles Debunne's work),
and on the realistic visualization (i.e. containing a lot of information)
of the surface and the changes in its aspect
(Raphael Heiss's work).
With Raphael, I work on real-time realistic rendering
of the liver surface and the instruments effects on it.
We dealt with three aspects:
- the grain of the skin;
- the reflects of the light source;
- the marks done by the instruments (e.g. cauterisation).
To be noted that the two first aspects are motivated
by the fact that texture and reflects streching
contain alot of information about the object relief.
These three aspects are turn into three OpenGL rendering passes,
by turning each case into textural operation:
- the skin grain should have a high resolution even when zooming.
So we use a tiled texture, which supposes to generate
a pattern (cyclical), and to map it with minimal distortions.
The pattern is generated using Voronoi diagrams,
the distortion minimisation use a spring-based relaxation. - the reflects can be dealt with using environment mapping;
we refresh the reflect texture that represents the light ring
which size depends on the distance and which blurryness
depends of the surface roughness. - the mark of the instruments in drawn in the third texture layer,
which is in bijection with the surface. The problem consists in
getting with texture location is under the mouse and how
much texture deformation exists there,
so that the spot can be drawn in the texture in such a way that
it appears regular when mapped.
With Jean-Christophe Lombardo and Marie-Paule Cani,
we published a method to
detect in real-time
the faces that are in contact with a tool,
by using the graphics hardware of SGIs
(OpenGL FEEDBACK render mode).
Pierre-Olivier Agliati has integrated the adaptive
deformation simulation model of Gilles Debunne,
the real-time collision detections,
the reflects of Raphael Heiss,
and the distortion-free textures.
With Franck Sénégas, we simulate the liver surface aspect
variation in reaction to the instruments:
contusions and blood drops, withening after compression,
cauterisation. We use for this animated textures.
Animation of drops on a cube:
(mpeg, 4.9 Mb).
Drops, cauterization, compression on a liver:
(mpeg, 25 Mb).
Principle:
- make triangular texture samples whose borders are compatible.
For liver skin, we use Worley procedural textures
(roughly, Voronoi diagram).
- build an equilateral mesh on the surface
(e.g. using Turk algorithm [Sig'92])
- project it on the surface using geodesics,
and parameterize the area of triangular texture patches.
- choose random boundary conditions on the texture mesh edges,
and map samples that are compatible with them.
The following image illustrates at which point reflects
provide an information about the relief:
in the present case, neither the shading, or the shadows,
or the perspective, or the depth of field
give any usable 3D information.
Moreover, reflects are very sensitive to small pressure changes,
thus provide an extra clue about the contact and the stress.