Panoramic Photography with Fisheye Lenses


by

Dan Slater

(Copyright 1995, All rights reserved)
Published in the IAPP Journal in 1996

Introduction

Scanning panoramic cameras have been used to produce pictures with a continuous 360° horizontal coverage. For example, the Hulcherama 120 panoramic camera when configured with a 35 mm focal length lens provides a continuous image coverage of 80° vertically and 1100° horizontally on a roll of 120 film. The 1100° horizontal coverage indicates that the camera can produce a single film frame containing slightly more than 3 rotations on a roll of film. There can be changes in the scene during the multiple rotations of the camera.

This paper describes a method of using a fisheye lens to produce panoramic images with a cylindrical perspective similar to those made by a scanning panoramic camera. In this process, one or more images are recorded with a camera using a hyperwide 220° fisheye lens. The resulting film frames are scanned into a computer and digitally resampled to form a cylindrical perspective panoramic image. The primary goal of this project was to develop a method of producing extreme wide angle panoramic images with a compact hand held camera. The intended application was action point of view (POV) and virtual reality (VR) photography.

Image recording

The cylindrical perspective panoramic image is formed by digital image processing of one or more hyperwide fisheye photographs. While not mandatory, it is highly desirable that the fisheye lens have a field of view of greater than 180° for reasons that will become apparent. There are relatively few lenses that meet this requirement:
 
Lens
Focal length
f#
Image diagonal
Field of view
Reflective ball
___
__
_______
360°
Nikkor fisheye
6 mm
2.8
23.0 mm
220°
Nikkor fisheye
6 mm
5.6
21.6 mm
220°
Zeiss V1936
16 mm
6.8
59.0 mm
210°
Volpi Peri-Apollar
25 mm
4.0
12.0 mm
360°x60°
Volpi Peri-Apollar
40 mm
6.0
20.0 mm
360°x60°

One of the main goals of this project was a compact point of view (POV) camera setup suitable for fast and unobtrusive hand-held panoramic photography. The hyperwide Nikon 6 mm f5.6 220° fisheye lens provided the best solution in terms of image quality, size, weight and cost. This lens has protruding rear elements that requires a camera with a mirror lockup capability. Because of the extremely wide angle of view, care is required to insure that parts of the camera do not protrude into the frame. When used with a Nikon F4 camera, the larger battery pack should be used as it protrudes less into the image than the smaller battery pack. Some of the aspheric reflector designs (Manley, 1991) are also worthy of further consideration as these can provide a large image size and very wide field of view. The Volpi Peri-Apollar lenses have a more limited coverage area (+30°, -30° to horizon) than the Nikon lenses (+90°, -20° to horizon).

In the simplest case, the fisheye lens is pointed straight up toward the zenith. A camera equipped with the 6 mm Nikon fisheye lens will photograph everything from an angle 20° below the horizon to the zenith. If more than one shot is taken in rapid succession, one can duplicate the image changes that occur during the multiple rotations of a scanning panoramic camera. Strobe lighting can be used for scene illumination as all parts of the frame are exposed simultaneously. Because a wide angle panoramic camera image can contain a considerable amount of detail, an emphasis should be placed on taking the sharpest possible photograph.

Image processing

Computer software is used to resample the spherical image produced by the fisheye lens into the cylindrical perspective as produced by scanning panoramic cameras. One method is:

1. Scan the photograph(s) into a computer at a high resolution. The image should be square and accurately centered in the frame so that the origin of the following polar to rectangular conversion step will be correctly centered. If the fisheye lens was tilted, center the scan on the zenith or nadir of the image. Some coverage below the horizon will be lost in this case. The scanner exposure and frame size settings should be identical for all scans.

Original fisheye photograph scanned in step 1.

2. Convert the image from the spherical fisheye format into the cylindrical perspective through the use of Adobe Photoshop or a similar digital image processing program (Snyder, 1983). Use the polar to rectangular conversion function with the best available interpolator, typically a bicubic spline (Wolberg, 1990).

Image after step 2.

3. Change the picture aspect ratio by the ratio of the lens vertical half angle field of view to the 360° horizontal field of view to eliminate anamorphic picture distortion near the horizontal plane. For example when using the Nikon 6 mm 220° field of view lens, the aspect ratio would be 360/110 or 3.27:1. There will be increasing distortion evident toward the top of the frame. This is because the vertical axis is now using an f theta mapping instead of the desired f tan theta mapping of a panoramic camera. Because of the spherical geometry, the zenith point must be mapped into a line at the top of the frame. This step assumes the lens has an f theta mapping, typical of most fisheye lenses but not true of a ball reflector.

Image after step 3 -- Notice the large amount of distortion toward the top of the frame. The bottom half of the frame is relatively undistorted. The distortion will be corrected in step 6.

4. The Nikon 6 mm f5.6 lens has a slight amount of chromatic aberration. While this effect is minor (1 or 2 pixels), correcting it will often noticeably improve the final result. The effect in the cylindrically resampled scene is that the image height of the red, green and blue components are different. The chromatic aberration can be suppressed by separate non-linearly vertical scaling of the red, and blue channels relative to the green channel.

5. Double the horizontal width of the frame leaving the current image at one side of the frame. Either process a second fisheye frame repeating steps 1 - 4 or copy the current image to the other side of the full frame. The image will now correspond to a 720° rotation. Additional rotations can be added in a similar manner. Some care will be needed to splice the image in a region of no motion when a second photograph is used.

6. The image geometry is fairly accurate from the bottom of the frame to 30° above the horizon line. At higher elevation angles, it becomes necessary to approximate the vertical f tan theta mapping of a scanning panoramic camera. This nonlinear vertical correction can be done in Adobe Photoshop by a displacement map or approximated with the vertical spherize function.

7. Apply any brightness, color and cropping enhancements as needed by the scene. Unsharp masking can be used to improve the apparent scene sharpness and can be quite useful because of the small size of the original fisheye film frames.

This panoramic photograph taken at Caesar's Palace in Las Vegas was derived by digitally resampling a single Nikon 220° fisheye image. This image has a horizontal coverage of 720° (2 rotations) and a vertical coverage from 20° below the horizon to nearly 70° above the horizon line.

Full sphere images

Two or more hyperwide fisheye photographs can be combined providing a single image covering 4pi steradians (all possible viewing angles). The image is produced from photographs taken with a pair of back to back fisheye cameras providing zenith and nadir views or simply by moving a single camera between the two positions.

This camera system was built for 4pi steradian point of view and virtual reality photography applications. This system consists of a pair of 220° Nikkor fisheye lenses mounted on opposing motor driven Nikon F camera bodies. The camera system is designed to be compact, rugged, minimum weight and remotely operable. It completely records the scene in all possible viewing directions (4pi steradians) from a given location.

Ideally, the fisheye lens perspective center (lens entrance pupil) should be at the same location for both the zenith and nadir shots. An additional nadir photograph with tripod legs or other camera supports moved to a different position can be used to remove these items from the final photograph. All exposures should be matched. The images are combined by adding the following steps after the correspondingly numbered steps in the previous image processing procedure.

1A. If a tripod or other camera support needs to be removed from the scene, align the two nadir fisheye views. In Photoshop combine the two views as separate layers. Invert the brightness of the reference scene and translate / rotate the upper layer until the image is aligned. The image will turn gray when the images are aligned. Invert the lower negative image back to positive. Add a masking layer with a default enable of the lower layer. Brush out the camera support in the matte. Merge (flatten) the image into one layer when this step is completed.

5A. This step will combine the lower and upper cylindrically mapped fisheye images leaving an image with a 180° vertical span and a 360° horizontal span. The procedure is: Increase the height of the upper image frame to correspond to a 180° vertical span, leaving the bottom part blank. The frame height should be 1/4 of the 720° horizontal span. Flip the lower image so that it is correctly oriented and then move it into the frame as a new layer. Align the lower image by setting the transparency to 50% and moving the layer as required Turn off the upper layer transparency and create a matte layer to merge the lower and upper images. Apply any necessary color correction to match the lower and upper image segments. Flatten the image into a single layer and crop out a complete 360° azimuth segment.

There are several options at this point. You can produce images corresponding to zenith or nadir images from a synthetic 360° fisheye (4pi steradian) lens and/or create a multiturn panoramic image. The 4pi steradian zenith and nadir images, shown below, are created by a rectangular to polar conversion of a normal or inverted cylinder image (output of step 5A). Alternately, an ultrawide cylindrical panoramic image can be created from the output of step 5A. Follow step 5A with a repeat of step 5 and then by steps 6 and 7.

These photographs are a simulation of a 360 degree field of view fisheye lens that covers all possible viewing angles (4 pi steradians). These images are similar to reflection images seen in a silvered ball. A silvered ball however would have a 2f sin (theta/2) mapping instead of the f theta mapping of these images resulting in a large amount of compression toward the outside edge of the image (Ray, 1988, Snyder, 1983). The left photograph is a straight up (zenith) view. The photograph on the right is a straight down (nadir) view. The camera viewpoint height is approximately 3 feet above the ground. Both of these images were derived from a set of 3 photographs taken with a Nikon 220° fisheye lens. The images show a cluster of tall radio towers located at the top of Mount Wilson in California. Additional concentrically mirrored image rings could be added to show the optical effect of a fisheye lens with a larger than 360° coverage.

The previous photograph was created from the same set of fisheye photographs that were used to produce the two 4 pi steradian fisheye images. This photograph approximates a scanning panoramic camera image with a 720° horizontal and a 140° vertical field of view. The camera viewpoint is approximately 3 feet above the ground. There is some residual distortion in the vertical resampling process so that the vertical field of view is actually somewhat larger. If an equivalent photograph were to be taken with a conventional scanning panoramic camera using 120 format film, it would require using a lens with an 11 mm focal length.

Conclusions

All photographs for the preceding examples were taken with Nikon 6 mm f5.6 fisheye lenses. For the Las Vegas images, this lens provided coverage from 20° below the horizon to the zenith in a single frame. For the radio tower photographs, the Nikon 6 mm fisheye lens provided a wide 40° overlap region near the horizon. This overlap region becomes important for a dual camera setup having separated lens entrance pupil locations. The perspective of nearby objects will be different between the two images when the lens entrance pupils are not coincident. The wide overlap region provides some flexibility in the image splicing of the nearby objects. The Nikon lens is preferred to a spherical reflector because the fisheye lens f theta mapping requires less drastic image resampling, resulting in a sharper image. A correctly designed aspheric reflector could provide similar performance. There is no fundamental limit to the field of view of a fisheye lens that uses a standard f theta mapping function. In principle, a lens could be built that would have a 4pi steradian field of view, directly providing images similar to the composite images shown in this article.

In this article, the conversion from the spherical (fisheye lens) perspective to the cylindrical (scanning panoramic camera) perspective was accomplished by digital image processing. It is also possible to perform these transformations optically. For example, the spherical fisheye image can be converted to cylindrical form by using a hyperwide fisheye lens in a photographic enlarger to image onto a cylindrically wrapped paper easel. Alternately, a camera could be built consisting of back to back hyperwide fisheye lenses directly imaging onto a cylindrical focal plane. There is a strong similarity between these types of optical transformation techniques and the cylindrical anamorphic drawing toys originally developed in the sixteenth century (Turner, 1983).

The fisheye lens / digital image resampling technique provides an alternative to scanning panoramic cameras with a number of advantages:

  1. The camera setup is very compact and well suited to hand held point of view (POV) photography.
  2. The range of shutter speeds is larger than that generally obtainable with scanning cameras.
  3. Strobe fill illumination can be used since the entire film frame is exposed at one instant.
  4. High angle objects including the zenith can be in the shot.
  5. The vertical angle of view can greatly exceed the coverage obtainable with currently available scanning panoramic cameras if two opposing view fisheye frames are joined together.
A scanning panoramic camera such as the Hulcherama provides the following advantages:
  1. Much more scene detail can be recorded. The film frame is much larger. The lenses are sharper.
  2. No digital image processing is needed.
  3. Multiturn images are easier to form and have less restrictions.
  4. The scanning panoramic camera has better coverage below the horizontal plane (unless 2 fisheye frames are merged).
  5. The lens is less exposed and less easily damaged.

References

Manley, P., Unusual Telescopes, Cambridge University Press, Cambridge, England, (1991)

Includes a discussion on reflective ball optics.

Ray, S., Applied Photographic Optics, Imaging Systems for Photography, Film and Video, Focal Press, Boston, MA (1988)

A detailed technical discussion of perspective, lens mapping functions and fisheye lenses.

Snyder, J., Map Projections Used by the U. S. Geological Survey, US Government Printing Office, Washington, DC (1983)

A detailed mathematical description of various map projection functions including cylindrical and spherical projections.

Turner, G., Nineteenth-Century Scientific Instruments, University of California Press, Los Angeles, CA (1983)

A book describing early scientific instruments. Includes a description of anamorphic toys that optically undistort a distorted image.

Wolberg, G., Digital Image Warping, IEEE Computer Society Press, Los Alamitos, CA (1990)

An excellent although quite technical book on the theory of image resampling.

Home