Seite 16 - Boards & Solution/ECE Magazine July 2013

EMBEDDED DESIGN
July 2013 16
n Video image signal processing (ISP) has
come a long way from the time of analog sig-
nals. Today, digital signal processing makes
possible image data manipulation at the bit
level, offering unprecedented control over
image quality. Digital signal processing, of
course, is not the same as a digital signal
processor, or DSP.While DSP has been popular
for video image signal processing in the digital
domain, ISP can be implemented by a number
of processing devices: DSPs, ASICs, ASSPs and,
increasingly, field programmable gate arrays,
or FPGAs.
There are several reasons for the growing pop-
ularity of FPGAs. Two of those reasons reflect
recent trends in security cameras that dramat-
ically increase the quantity of image data that
needs to be processed, and the third is an eco-
nomic reason – the cost of the camera bill of
materials (BOM). There are two major trends
that are changing the way security cameras
are architected: the advent of megapixel sensors,
and the need for high (or wide) dynamic
range (HDR/WDR)
There was a time when a VGA resolution
sensor was sufficient for security camera pur-
poses, usually viewed by an operator or simply
archived for later review. However, with the
dramatic increase in the number of security
cameras used worldwide, there are not enough
human operators available and so the security
industry has begun to rely on software to ana-
lyze the video, either in real time or later, to
discover if anything untoward occurred in the
region of interest. Sophisticated video analytics
(VA) algorithms have been developed to high-
light the extraordinary from the ordinary;
however, in order to be effective, these algo-
rithms need much more detail that can be
provided by VGA resolution cameras. Cameras
need higher resolution for VA to be able to dis-
cern general movement in restricted and/or
large areas, e.g. a parking lot is at capacity. A
camera needs approximately 30 pixels/in for
license plate recognition, and approximately
150pixels/in to view more detailed activity,
such as identifying cash register transactions.
One megapixel covers the detail in a seven
foot by seven foot area, and it would take four
VGA cameras to match the power of a one
megapixel camera. Image sensors have been
developed, and are commercially available, for
one, two, five and even ten megapixel resolution.
Obviously, as the number of pixels has in-
creased, so too has the amount of data that
must be processed to take advantage of the
increased resolution.
High dynamic range (HDR) also known as
wide dynamic range (WDR), measures how
well the sensor and the ISP function see into
both dark and brightly lit areas. We are all
familiar with amateur outdoor family pictures
taken with the sun behind the people in the
photograph. While the landscape bathed in
sunlight is bright and clear, people faces are
quite dark. This happens because the (usually
automatic) camera adjusts its exposure to the
bright sunlight in the scene. That exposure,
however, is too short to properly register the
darker objects. If one manually sets the expo-
sure or aperture to let in more light, one will
be able to discern detail in the dark areas, but
at the expense of detail in the bright areas,
which now are overexposed and completely
washed out.
This is not a good result for either human op-
erators or for VA software, since much of the
detail in the region of interest is lost. HDR
sensors solve this problem in creative ways, all
of which depend on capturing multiple images,
each with different exposure times, and then
having the ISP pipeline combine and blend
these images to preserve and render visible de-
tail from both bright and dark areas in the re-
gion of interest. Obviously, multiple exposures
for the same image translate into an increased
amount of data to be processed. For example,
when a video camera that outputs full HD
1080p images at 60 frames per second is work-
ing with a HDR sensor that takes 3 exposures
per frame, the ISP pipeline inside the camera
is actually processing the equivalent of 60 x 3,
High dynamic range image signal
processing using FPGAs
By Niladri Roy, Lattice Semiconductor
This article shows
that low-cost,
low-power FPGAs
are suited to take on
the enormously increased
signal processing load caused
by the need to use megapixel
sensors and HDR in video
cameras for security
and surveillance.