U.S. patent application number 17/579712 was filed with the patent office on 2022-05-05 for stitched image.
The applicant listed for this patent is The Government of the United States, as represented by the Secretary of the Army, The Government of the United States, as represented by the Secretary of the Army. Invention is credited to Michael Badger, Dennis Bushmitch.
Application Number | 20220139077 17/579712 |
Document ID | / |
Family ID | 1000006093977 |
Filed Date | 2022-05-05 |
United States Patent
Application |
20220139077 |
Kind Code |
A1 |
Bushmitch; Dennis ; et
al. |
May 5, 2022 |
STITCHED IMAGE
Abstract
Various embodiments associated with a composite image are
described. In one embodiment, a handheld device comprises a launch
component configured to cause a launch of a projectile. The
projectile is configured to capture a plurality of images.
Individual images of the plurality of images are of different
segments of an area. The system also comprises an image stitch
component configured to stitch the plurality of images into a
composite image. The composite image is of a higher resolution than
a resolution of individual images of the plurality of images.
Inventors: |
Bushmitch; Dennis;
(Somerset, NJ) ; Badger; Michael; (Ocean Grove,
NJ) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
The Government of the United States, as represented by the
Secretary of the Army |
Washington |
DC |
US |
|
|
Family ID: |
1000006093977 |
Appl. No.: |
17/579712 |
Filed: |
January 20, 2022 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
15273924 |
Sep 23, 2016 |
11244160 |
|
|
17579712 |
|
|
|
|
13547352 |
Jul 12, 2012 |
9870504 |
|
|
15273924 |
|
|
|
|
Current U.S.
Class: |
382/103 |
Current CPC
Class: |
H04N 5/23293 20130101;
H04N 5/23238 20130101; G06V 10/16 20220101; B64C 2201/127 20130101;
H04N 7/185 20130101; G06T 3/4038 20130101; B64C 2201/08 20130101;
B64C 39/024 20130101; H04N 5/28 20130101; G06T 7/33 20170101; G06V
20/13 20220101; G06T 2207/30212 20130101; B64C 2201/123 20130101;
G06T 11/60 20130101; G06T 7/38 20170101; H04N 5/2624 20130101; G06V
20/52 20220101; G06T 2207/30181 20130101; G06T 2207/20221 20130101;
G06T 2207/10032 20130101; G06T 5/50 20130101; G06T 3/4053
20130101 |
International
Class: |
G06V 20/13 20060101
G06V020/13; G06T 11/60 20060101 G06T011/60; H04N 7/18 20060101
H04N007/18; G06T 7/33 20060101 G06T007/33; H04N 5/262 20060101
H04N005/262; H04N 5/232 20060101 H04N005/232; G06T 3/40 20060101
G06T003/40; G06V 20/52 20060101 G06V020/52; G06T 7/38 20060101
G06T007/38; B64C 39/02 20060101 B64C039/02; G06T 5/50 20060101
G06T005/50; H04N 5/28 20060101 H04N005/28 |
Goverment Interests
GOVERNMENT INTEREST
[0002] The innovation described herein may be manufactured, used,
imported, sold, and licensed by or for the Government of the United
States of America without the payment of any royalty thereon or
therefor.
Claims
1. A processor that is communicatively coupled to a non-transitory
computer-readable medium and that is configured to execute a
command set stored by the non-transitory computer-readable medium
to effectuate operation of a component set, the component set
comprising: comprising: a launch component configured to cause a
launch of a projectile from a handheld device, where the projectile
is configured to capture a plurality of images; an identification
component configured to identify a travel condition of the
projectile after launch; an instruction component configured to
instruct the projectile to capture the plurality of images in
response to identification of the travel condition; and a
compensation component configured to perform a compensation for a
discrepancy between at least two individual images of the plurality
of images, where the discrepancy is caused by movement of the
projectile, where the projectile comprises an accelerometer
configured to obtain a speed and vector information set for
individual images of the plurality of images and where the
discrepancy is ascertained from a difference among the speed and
vector information set for individual images of the plurality of
images.
2. The processor of claim 1, where the projectile is connected to
the handheld device by way of a physical link, where the projectile
transmits the plurality of images to the handheld device by way of
the physical link, where the physical link is proactively broken
after the projectile completes transmission of the plurality of
images to the handheld device.
3. The processor of claim 1, the component set comprising: a stitch
component configured to stitch at least part of the plurality of
images into a composite image, where the composite image covers a
view of about 360 degrees, where individual images of the plurality
of images cover a view of less than about 360 degrees, where an
overlap exists among at least some of the individual images of the
plurality of images, and where at least some of the overlap is used
to improve resolution of the composite image over the resolution of
the individual images of the plurality of images.
4. The processor of claim 1, where the processor is resident upon
the handheld device.
5. The handheld device of claim 2, where the physical link is an
optical physical link.
6. The processor of claim 1, where the processor is resident upon
the projectile.
7. The processor of claim 1, the component set comprising: a stitch
component configured to at least part of the plurality of images
into a composite image where the composite image covers a view of
about 360 degrees, where individual images of the plurality of
images cover a view of less than about 360 degrees, where an
overlap exists among at least some of the individual images of the
plurality of images, and where at least some of the overlap is used
to improve resolution of the composite image over the resolution of
the individual images of the plurality of images.
8. The processor of claim 1, where the plurality of images
comprises a first image and a second image and where the
discrepancy is caused by at least partially lateral movement of the
projectile between when the projectile captures the first image and
when the projectile captures the second image.
9. The processor of claim 1, the component set comprising: a stitch
component configured to stitch at least part of the plurality of
images into a composite image; a comparison component configured to
make a comparison between the composite image against a retained
image, where the comparison produces a comparison result and where
the comparison result indicates at least one difference between the
composite image and the retained image; and an update component
configured to cause an update upon the retained image, where the
update is a modification of the retained image such that the at
least one difference is eliminated.
10. The processor of claim 1, the component set comprising: a
stitch component configured to stitch at least part of the
plurality of images into a composite image; a comparison component
configured to make a comparison between the composite image against
a retained image, where the comparison finds a common feature set
among the composite image and the retained image; an image
alignment component configured to make an aligned image from the
composite image and the retained image through use of the common
feature set; and an interface component configured to cause an
interface to be presented that enables a user to access location
information from the aligned image, where the retained image is not
obtained by the projectile.
11-20. (canceled)
21. A non-transitory computer-readable medium that is
communicatively coupled to a processor and that is configured to
store a command set executable by the processor to effectuate
operation of a component set, the component set comprising:
comprising: a launch component configured to cause a launch of a
projectile from a handheld device, where the projectile is
configured to capture a plurality of images; an identification
component configured to identify a travel condition of the
projectile after launch; an instruction component configured to
instruct the projectile to capture the plurality of images in
response to identification of the travel condition; and a
compensation component configured to perform a compensation for a
discrepancy between at least two individual images of the plurality
of images, where the discrepancy is caused by movement of the
projectile, where the projectile comprises an accelerometer
configured to obtain a speed and vector information set for
individual images of the plurality of images and where the
discrepancy is ascertained from a difference among the speed and
vector information set for individual images of the plurality of
images.
22. The non-transitory computer-readable medium of claim 21, where
the projectile is connected to the handheld device by way of a
physical link, where the projectile transmits the plurality of
images to the handheld device by way of the physical link, where
the physical link is proactively broken after the projectile
completes transmission of the plurality of images to the handheld
device.
23. The non-transitory computer-readable medium of claim 22, where
the physical link is proactively broken after the projectile
completes transmission of the plurality of images to the handheld
device.
24. The non-transitory computer-readable medium of claim 21, the
component set comprising: a stitch component configured to stitch
at least part of the plurality of images into a composite image,
where the composite image covers a view of about 360 degrees, where
individual images of the plurality of images cover a view of less
than about 360 degrees, where an overlap exists among at least some
of the individual images of the plurality of images, and where at
least some of the overlap is used to improve resolution of the
composite image over the resolution of the individual images of the
plurality of images.
25. The non-transitory computer-readable medium of claim 21, where
the non-transitory computer-readable medium is resident upon the
handheld device.
26. The non-transitory computer-readable medium of claim 21, where
the non-transitory computer-readable medium is resident upon the
projectile.
27. The non-transitory computer-readable medium of claim 21, the
component set comprising: a stitch component configured to at least
part of the plurality of images into a composite image where the
composite image covers a view of about 360 degrees, where
individual images of the plurality of images cover a view of less
than about 360 degrees, where an overlap exists among at least some
of the individual images of the plurality of images, and where at
least some of the overlap is used to improve resolution of the
composite image over the resolution of the individual images of the
plurality of images.
28. The non-transitory computer-readable medium of claim 21, where
the plurality of images comprises a first image and a second image
and where the discrepancy is caused by at least partially lateral
movement of the projectile between when the projectile captures the
first image and when the projectile captures the second image.
29. The non-transitory computer-readable medium of claim 21, the
component set comprising: a stitch component configured to stitch
at least part of the plurality of images into a composite image; a
comparison component configured to make a comparison between the
composite image against a retained image, where the comparison
produces a comparison result and where the comparison result
indicates at least one difference between the composite image and
the retained image; and an update component configured to cause an
update upon the retained image, where the update is a modification
of the retained image such that the at least one difference is
eliminated.
30. The non-transitory computer-readable medium of claim 21, the
component set comprising: a stitch component configured to stitch
at least part of the plurality of images into a composite image; a
comparison component configured to make a comparison between the
composite image against a retained image, where the comparison
finds a common feature set among the composite image and the
retained image; an image alignment component configured to make an
aligned image from the composite image and the retained image
through use of the common feature set; and an interface component
configured to cause an interface to be presented that enables a
user to access location information from the aligned image, where
the retained image is not obtained by the projectile.
Description
CROSS-REFERENCE
[0001] This application is a continuation application of, and
claims priority to, U.S. patent application Ser. No. 15/273,924
that has a U.S. Pat. No. 11,244,160 with an issue date of Feb. 8,
2022. U.S. patent application Ser. No. 15/273,924 is a divisional
application of, and this application also claims priority to, U.S.
patent application Ser. No. 13/547,352 that has a U.S. Pat. No.
9,870,504 with an issue date of Jan. 16, 2018. The entirety of U.S.
patent application Ser. No. 13/547,352 and the entirety of U.S.
patent application Ser. No. 15/273,924 are incorporated by
reference.
BACKGROUND OF THE INVENTION
[0003] In a combat context, an aerial image of a Warfighter's
geographical locale can be used to help soldiers to better
understand their surroundings, make tactical decisions, identify an
enemy, and the like. The image can be taken by an airplane or an
aerial drone and this aerial image can be further relayed to a
soldier by means of radios and mobile computing platforms. The
solider can use information provided in the aerial image to make
critical warfighting and life preservation decisions. For example,
the aerial image can show terrain information. Based on this
terrain information, the soldier can create a travel path to lower
travel difficulty, determine a preferable location for possible
enemy engagement, etc. Thus, the aerial image is used to help the
soldier. An expensive and sophisticated aerial drone systems can be
used, but these systems can be difficult to operate, relatively
large in size and weight, and therefore limit their battlefield
proliferation amongst military personnel.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] Incorporated herein are drawings that constitute a part of
the specification and illustrate embodiments of the detailed
description. The detailed description will now be described further
with reference to the accompanying drawings as follows:
[0005] FIG. 1 illustrates one embodiment of a handheld device with
a launch component and an image stitch component;
[0006] FIG. 2 illustrates one embodiment of a plurality of images
with an area of interest;
[0007] FIG. 3 illustrates one embodiment of a composite image with
the area of interest;
[0008] FIG. 4 illustrates one embodiment of a handheld device
comprising the launch component, the image stitch component, and a
global positioning system component;
[0009] FIG. 5 illustrates one embodiment of a handheld device
comprising the launch component, a compensation component, and an
image stitch component;
[0010] FIG. 6 illustrates one embodiment of a handheld device
comprising the launch component, an identification component, an
instruction component, and an image stitch component;
[0011] FIG. 7 illustrates one embodiment of a handheld device
comprising the launch component, the image stitch component, a
comparison component, an image alignment component, and an
interface component;
[0012] FIG. 8 illustrates one embodiment of a handheld device
comprising the launch component, the comparison component, an
update component, and the image stitch component;
[0013] FIG. 9 illustrates one embodiment of the projectile
comprising a housing, a payload, and a pair of gas chambers;
[0014] FIG. 10 illustrates one embodiment of a system comprising an
access component and an alignment component;
[0015] FIG. 11 illustrates one embodiment of a system comprising
the access component, the alignment component, a comparison
component, an analysis component, and a feature component;
[0016] FIG. 12 illustrates one embodiment of a method; and
[0017] FIG. 13 illustrates one embodiment of a system comprising a
processor and a computer-readable medium.
DETAILED DESCRIPTION
[0018] Systems, methods and other embodiments disclosed herein are
related to using a stitched image to help the solider. As discussed
in the background, the soldier can use the aerial image to gain
information. However, the aerial image can quickly become outdated
and therefore while some information can be useful, such as terrain
information, other information can be less helpful or even
detrimental, such as enemy soldier location.
[0019] To obtain up-to-date information, the soldier can employ a
small projectile that is launched in the air and that obtains
images. For example, six images of sixty degrees can be taken with
slight overlap by the small projectile. The six images can be
stitched together to form a stitched image covering 360 degrees.
The stitched image covering 360 degrees can be displayed on an
interface of a handheld device of the solider or on a separate
interface (e.g., displayed on a screen of a laptop computer).
Therefore, the soldier can quickly obtain up-to-date information
about his surroundings.
[0020] In one embodiment, the stitched image can be overlaid with
the previously obtained and pre-stored aerial image/mobile map
image on the device. Thus, information from the stitched image and
the pre-stored aerial image can be used together by the soldier
(e.g., used simultaneously) to enhance the overall information
delivered to the soldier. For example, the soldier can hold a
handheld device that retains the previously obtained aerial image
in memory. The aerial image can be processed before being loaded on
the handheld device with information being added to the aerial
image, such as identifiers, elevation information, etc. The
stitched image can be aligned with the aerial image such that the
soldier can simultaneously use information of both images. In one
example, an enemy position can be illustrated at a location in the
stitched image and the soldier can obtain elevation information of
the location through the aerial image. Thus, the soldier can
benefit for using both real time information and pre-processed
information together. In another example, global positioning data
embedded in the pre-stored aerial image/mobile map image can be
used to determine the coordinates of stitched image identified data
elements.
[0021] The following includes definitions of selected terms
employed herein. The definitions include various examples. The
examples are not intended to be limiting.
[0022] "One embodiment", "an embodiment", "one example", "an
example", and so on, indicate that the embodiment(s) or example(s)
can include a particular feature, structure, characteristic,
property, or element, but that not every embodiment or example
necessarily includes that particular feature, structure,
characteristic, property or element. Furthermore, repeated use of
the phrase "in one embodiment" may or may not refer to the same
embodiment.
[0023] "Computer-readable medium", as used herein, refers to a
medium that stores signals, instructions and/or data. Examples of a
computer-readable medium include, but are not limited to,
non-volatile media and volatile media. Non-volatile media may
include, for example, optical disks, magnetic disks, and so on.
Volatile media may include, for example, semiconductor memories,
dynamic memory, and so on. Common forms of a computer-readable
medium may include, but are not limited to, a floppy disk, a
flexible disk, a hard disk, a magnetic tape, other magnetic medium,
other optical medium, a Random Access Memory (RAM), a Read-Only
Memory (ROM), a memory chip or card, a memory stick, and other
media from which a computer, a processor or other electronic device
can read. In one embodiment, the computer-readable medium is a
non-transitory computer-readable medium.
[0024] "Component", as used herein, includes but is not limited to
hardware, firmware, software stored on a computer-readable medium
or in execution on a machine, and/or combinations of each to
perform a function(s) or an action(s), and/or to cause a function
or action from another component, method, and/or system. Component
may include a software controlled microprocessor, a discrete
component, an analog circuit, a digital circuit, a programmed logic
device, a memory device containing instructions, and so on. Where
multiple components are described, it may be possible to
incorporate the multiple components into one physical component or
conversely, where a single component is described, it may be
possible to distribute that single logical component between
multiple components.
[0025] "Software", as used herein, includes but is not limited to,
one or more executable instructions stored on a computer-readable
medium that cause a computer, processor, or other electronic device
to perform functions, actions and/or behave in a desired manner.
The instructions may be embodied in various forms including
routines, algorithms, modules, methods, threads, and/or programs
including separate applications or code from dynamically linked
libraries.
[0026] FIG. 1 illustrates one embodiment of a handheld device 100
with a launch component 105 and an image stitch component 110. The
launch component 105 is configured to cause a launch of a
projectile 115. For example, the projectile 115 (e.g., image sensor
projectile) can be launched from the handheld device 100. In one
embodiment, the handheld device 100 is a smart phone, personal
electronic device, laptop computer, etc. However, the projectile
115 can be considered a handheld device itself and thus comprise
the launch component 105, the image stitch component 110, or other
component disclosed herein.
[0027] The projectile 115 is configured to capture a plurality of
images, where individual images of the plurality of images are of
different segments of an area. For example, the plurality of images
covers a range of about 360 degrees such that eight individual
images cover about 45 degrees each with some overlap among
individual images of the plurality of images, which can be seen in
greater detail with FIG. 2. Thus, this is one example of where the
composite image covers a view of about 360 degrees and individual
images of the plurality of images cover a view of less than about
360 degrees.
[0028] The image stitch component 110 is configured to stitch the
plurality of images into a composite image. In one embodiment, the
composite image creates a seamless 360-degree image of an area. The
composite image is of a higher resolution than a resolution of
individual images of the plurality of images captured by the
projectile 115. Multiple slices of the same area or overlapping
area can also improve contrast and/or color balance of a target
resolution of the composite image. The composite image can be an
enhanced version of what could be obtained with a singular image.
Thus, combining a plurality of images into a composite image can be
of greater enhancement than the individual images of the plurality
of images.
[0029] FIG. 2 illustrates one embodiment of a plurality of images
200-235 with an area of interest 240. The plurality of images
200-235 can be the plurality of images captured by the projectile
115 of FIG. 1. Images discussed herein can include photographs,
infrared images, radar images, and others. The plurality of images
can overlap one another such that an overlap exists among at least
some of the individual images. For example, the dark links
separating the individual images 200-235 can be overlapping areas.
The overlap can cover at least part of the area of interest 240 and
at least some of the overlap is used to improve resolution of the
composite image over the plurality of images 200-235.
[0030] In one embodiment, the projectile 115 of FIG. 1 can rotate
and while the projectile 115 of FIG. 1 rotates a camera of the
projectile captures the individual images 200-235. In one example,
the projectile 115 of FIG. 1 can launch without rotation. When a
particular launch condition is met, the projectile 115 of FIG. 1
can deploy wings that cause the projectile 115 of FIG. 1 to rotate.
As rotation occurs, the camera can capture the individual images
200-235. Once the individual images 200-235 are captured and sent
to the handheld device 100 of FIG. 1, the wings can be removed from
the projectile 115 of FIG. 1, remain, etc. While eight individual
images 200-235 are illustrated, it is to be appreciated by one of
ordinary skill in the art that more or less than eight images can
be the plurality of images with or without overlap (e.g., twelve
images with overlap captured over a couple of seconds).
[0031] FIG. 3 illustrates one embodiment of a composite image 300
with the area of interest 240. The image stitch component 110 of
FIG. 1 can create the composite image 300 from the plurality of
images 200-235 of FIG. 2. The plurality of images 200-235 of FIG. 2
can be captured by the projectile 115 of FIG. 1.
[0032] FIG. 4 illustrates one embodiment of a handheld device 400
comprising the launch component 105, the image stitch component
110, and a global positioning system (GPS) component 405. The GPS
component 405 is configured to provide position information (e.g.,
provide current position information on a display of the handheld
device 400). The handheld device 400 can use the launch component
105 to cause the projectile 115 to launch.
[0033] The projectile 115 is connected to the handheld device 400
by way of a link. 410. In one embodiment, the link 410 is a
wireless communication link, where the plurality of images can be
sent securely to the handheld device 400 from the projectile 115.
In one embodiment, the link 410 is a physical link and as such the
projectile 115 is configured to be tethered by the physical link to
the handheld device 400 when the launch of the projectile 115
occurs. In one embodiment, the projectile 115 is configured to
transmit the plurality of images to the handheld device 400 by way
of the physical link. Prior to transmission, the projectile 115 can
be configured to compress and/or encrypt the plurality of images
and then the handheld device 400 can uncompress and/or decrypt the
plurality of images before the image stitch component 110 operates
on the plurality of images. The physical link can be an optical
physical link, an electrical conductor, or other type of physical
link. In one embodiment, the physical link employs a serialized
digital image interface (e.g., over twisted pair). The power supply
can be supplied to the projectile 115 by way of the physical link.
The physical link can enable the projectile 115 to send information
to the handheld device 400 without concern of a transmission being
jammed or intercepted, which could occur with a wireless
signal.
[0034] In one embodiment, the projectile 115 is a first projectile.
The launch component 105 is configured to be loaded with the
projectile 115. The physical link between the first projectile and
the handheld device is broken after the projectile 115 transmits
(e.g., successfully transmits) the plurality of images to the
handheld device 400 upon descent of the projectile 115. The
handheld device 400 is configured to be loaded with a second
projectile after the physical link with the projectile 115 is
broken. In one embodiment, the physical link is proactively broken
after the projectile 115 completes transmission of the plurality of
images to the handheld device 400. The handheld device 400 (e.g.,
via software) or the projectile 115 (e.g., via software that
measures its flight pattern) can identify when the final individual
image of the plurality of images is successfully received and then
the handheld device 400 of FIG. 4 can cause the link 410 to be
severed (e.g., without user instruction).
[0035] The handheld device 400 can include a display screen
interface that displays the composite image. GPS capabilities of
the GPS component 405 can be used to provide location information,
where the location information can be displayed against the visual
elements of the composite image. The handheld device 400 can enable
the user to zoom into the composite image, pan the composite image,
etc.
[0036] FIG. 5 illustrates one embodiment of a handheld device 500
comprising the launch component 105, a compensation component 505,
and an image stitch component 110. When the projectile 115 is
launched, a camera (e.g., a simple pin-hole camera, a daytime
camera and a nighttime camera where individual images of the
plurality of images are taken exclusively from the daytime camera
or exclusively from the nighttime camera, etc.) of the projectile
115 can capture images in series. Thus, a location of the
projectile 115 when an earlier image is captured can be different
from a location of the projectile 115 when a latter image is
captured. A location difference can occur due to rise/fall of the
projectile 115, flight path of the projectile 115, wind moving the
projectile 115, and others. The location difference can change
perspective of the images, such as a different capture angle,
change visibility of images, etc. These changes can lead to a
discrepancy between individual images.
[0037] The compensation component 505 is configured to perform a
compensation for a discrepancy between at least two individual
images of the plurality of images, where the discrepancy is caused
by movement of the projectile 115. A result of this compensation is
used by the image stitch component 110 to produce the composite
image and thus the compensation component 505 aids the image stitch
component 110. In one embodiment, the plurality of images comprises
a first image and a second image and the discrepancy is caused by
at least partially lateral movement of the projectile between when
the projectile 115 captures the first image and when the projectile
captures the second image. This lateral movement of the projectile
115 can occur due to wind, launch angle, etc. In one embodiment,
the projectile 115 comprises an accelerometer configured to obtain
a speed and vector information set for individual images of the
plurality of images. The discrepancy can be observed through a
difference among the speed and vector information set for
individual images of the plurality of images. In one embodiment,
the compensation component 505 is part of the projectile 115 and
the plurality of images in a compensated form is sent to the
handheld device 500.
[0038] FIG. 6 illustrates one embodiment of a handheld device 600
comprising the launch component 105, an identification component
605, an instruction component 610, and an image stitch component
110. The identification component 605 is configured to identify a
travel condition of the projectile 115 after launch. The
instruction component 610 is configured to instruct the projectile
115 to take the plurality of images in response to identification
of the travel condition.
[0039] In one example, the launch component 105 causes the
projectile 115 to launch. The identification component 605 can
monitor flight of the projectile 115. A fuel engine of the
projectile 115 can separate from the projectile 115. The
identification component 605 can identify this separation. The
instruction component 610 can include a logic command that when
fuel engine separation occurs the projection should start acquiring
the plurality of images. The instruction component 610 can follow
this command and instruct the projectile 115 to begin capturing the
images. The projectile 115 can capture the plurality of images,
send the plurality of images to the handheld device 600, and the
image stitch component 110 can produce the compound image from the
plurality of images.
[0040] In one example, the handheld device 600 can include an
interface (e.g., monitor and keyboard) and a user can input an
order by way of the interface for the projectile 115 to launch. In
response to the order, the launch component 105 can cause the
projectile 115 to launch in a skyward direction or other direction.
The projectile 115 can travel in the air until the projectile 115
reaches its apogee (top of relatively parabolic flight) and then
descends towards ground. The identification component 605 can
monitor travel of the projectile 115 and determine when the apogee
is reached (e.g., actually reached, about to be reached, etc.). For
example, the identification component 605 can read output of a
fixed accelerometer of the projectile 115 and from this reading
determine when the apogee is reached. When this determination
occurs, the instruction component 610 can instruct the camera of
the projectile 115 to capture images, instruct how to capture
images (e.g., exposure length, camera angle, etc.), etc. The
captured images can be stitched together by the image stitch
component 110 to form the composite image. The composite image can
be displayed on a display of the handheld device 600.
[0041] FIG. 7 illustrates one embodiment of a handheld device 700
comprising the launch component 105, the image stitch component
110, a comparison component 705, an image alignment component 710,
and an interface component 715. The launch component 105 causes the
projectile 115 to launch (e.g., from the handheld device 700, from
a unit in direct communication with the handheld device 700, etc.).
The projectile 115 captures the plurality of images and sends the
plurality of images to the handheld device 700. The image stitch
component 110 creates the composite image from the plurality of
images.
[0042] The comparison component 705 can be configured to make a
comparison between the composite image and a retained image. The
retained image can be a non-real time image of an area where at
least part of the area of the non-real time image overlaps with at
least part of an area of the composite image. For example, the
retained image is an aerial image of a specific area stored in
memory of the handheld device 700 and the composite image is a real
time image of the specific area. In one embodiment, the retained
image is a real time image (e.g., up-to-date satellite image). The
retained image can be sent from another unit (e.g., airplane) and
retained in a memory of the handheld device 700 while the composite
image can also be retained in the memory. The comparison component
705 accesses the retained image in the memory to perform the
comparison between the composite image against the retained image.
The comparison finds common feature set (e.g., one or more common
feature) among the composite image and the retained image.
[0043] The image alignment component 710 is configured to make an
aligned image from the composite image and the retained image
through use of the common feature set found by the comparison
component 705. For example, the comparison component 705 can
identify a uniquely shaped rock formation in the retained image and
the composite image. The alignment component 710 can use the
identified formation to align the retained image with the composite
image (e.g., overlay the retained image with the composite image
through overlapping formation portions of the retained image and
the composite image). The retained image aligned with the composite
image forms the aligned image.
[0044] In one embodiment, the image alignment component 710 is
configured to update stored mapping data for future use with fresh
imagery. The comparison component 705 is configured to determine
displacement of the stitched image and the offline stored image.
When the offline image possesses geospatial coordinates, the
coordinates of visual elements of the stitched image thus can be
found, by way of the comparison component 705, by comparing the
stitched image with the offline image and then transferring the
geospatial coordinates from the offline image to the stitched
image.
[0045] The interface component 715 is configured to cause an
interface to be presented (e.g., an image displayed on the
interface) that enables a user to access location information from
the aligned image. For example, the retained image can be
pre-processed to include coordinate information (e.g., latitude and
longitude information) that becomes part of the aligned image. The
composite image can illustrate an enemy troop location that is
illustrated in the aligned image. The user can designate the enemy
troop location through the interface and the interface can present
on the aligned image coordinate information for the enemy troop
location. Therefore, a soldier can gain a combined benefit of
information from a real time image obtained from the projectile 115
and information of a preprocessed image.
[0046] FIG. 8 illustrates one embodiment of a handheld device 800
comprising the launch component 105, the comparison component 705,
an update component 805, and the image stitch component 110. The
comparison component 705 can be configured to make a comparison
between the composite image against the retained image, where the
comparison produces a comparison result and where the comparison
result indicates at least one difference between the composite
image and the retained image. The update component 805 is
configured to cause an update upon the retained image, where the
update is a modification of the retained image such that the at
least one difference is eliminated.
[0047] In one example, an airplane can take photographs of a
specified area and these photographs can be the retained image that
is retained in storage. These photographs can become out-of-date,
be missing information (e.g., due to cloud cover), etc. When the
composite image is created of the specified area, the composite
image can be considered more up-to-date than the retained image,
more accurate than the retained image, etc. The comparison
component 705 can identify differences between the retained image
and the composite image and the update component 805 can update the
retained image. In one embodiment, updating occurs when the
retained image is modified with information from the composite
image. In one embodiment, updating occurs when the retained image
is replaced by the composite image and thus the composite image
becomes the retained image. In one embodiment, the update component
805 causes the updated by sending an update instruction to a
central repository of images that holds the retained image. In one
embodiment, the projectile 115 sends the plurality of images and/or
the composite image to the central repository of images along with
the update instruction.
[0048] FIG. 9 illustrates one embodiment of the projectile 115
comprising a housing 905, a payload 910, and a pair of gas chambers
915. While the pair of gas chambers 915 is shown, other
configurations are possible such as a single rocket engine. The
projectile 115 can be attached to a mobile controller performing
image stitching (e.g., a handheld device disclosed herein) by way
of a coiled tether that uncoils as the projectile 115 travels. This
attachment can include a physical attachment as well as being
communicatively attached such that transmission of the plurality of
images can occur wirelessly (e.g., by way of Wi-Fi). The mobile
controller or the housing 905 can include a blade to cut the tether
once images are captured. The housing 905 can incorporate features
that assist in flight of the projectile 115. For example, the
housing 905 can have a point to allow for a maximum upward
trajectory (e.g., a higher trajectory can enable the projectile 115
to reach over high trees), the housing 905 can have wings (e.g.,
deployable wings (e.g., deployed when a certain condition is met,
such as speed), fixed wings, etc.) to provide a wobble of the
projectile 115 or to stabilize the projectile 115.
[0049] A payload 910 of the projectile 115 can include an image
capture component (e.g., the camera (e.g., visible light camera,
infrared camera, etc.)), at least one sensor (e.g., video sensor),
infrared illuminator, digital storage, a microprocessor (e.g.,
configured to compress captured images prior to their transmission
to the handheld device 100 of FIG. 1, where the handheld device 100
of FIG. 1 is configured to decompress the captured images and the
create the composite image), a receiver, a transmitter, a
self-destruct mechanism, etc. The payload 910 can include multiple
image capture components to increase resolution (e.g., each image
capture component facing a different direction) and can include
multiple video sensors to yield quicker system response and faster
stitching by the image stitch component 110 of FIG. 1. A sensor of
the payload 910 can identify when a certain situation occurs (e.g.,
a top portion of flight of the projectile 115). In response to the
certain situation occurring, the payload 910 can use the image
capture component to capture the plurality of images. The payload
910 can send the plurality of images captured to the handheld
device 100 of FIG. 1. The handheld device 100 of FIG. 1 can send a
confirmation message to the projectile 115 that the plurality of
images is successfully received and in response to receiving the
confirmation message, the payload 910 can destroy at least one
component of the projectile 115 (e.g., so hardware or software does
not fall into enemy hands). In one embodiment, the payload 910 can
be selected by a user (e.g., the user places at least some payload
component in the payload 910 prior to loading the projectile 115 in
the handheld device 100 of FIG. 1, user selects payload components
that are loaded from the handheld device 100 of FIG. 1 to the
payload 910 while the projectile 115 is loaded in the handheld
device 100 of FIG. 1, etc.).
[0050] The payload 910 can include other features to enhance
capabilities of the projectile 115. In one example, the payload 910
includes weather measurement instruments, where weather information
is captured and sent to the handheld device 100 of FIG. 1. In one
example, the payload 910 measures performance of the projectile 115
(e.g., projectile speed, information related to the projectile 115
experiencing a failure, etc.). Performance information of the
projectile 115 can be sent to the handheld device 100 of FIG. 1 and
be evaluated by a user for use in subsequent launches.
[0051] The launch component 105 of FIG. 1 can cause the projectile
115 to launch in order to collect images. The handheld device 100
of FIG. 1 can launch multiple projectiles consecutively,
concurrently, at different times, etc. In one embodiment, a first
projectile is launched with a first payload. The first payload can
include weather measurement instruments that measure weather
information. The measured weather information along with
performance data of the first projectile can be sent to the
handheld device 100 of FIG. 1. A processor of the handheld device
can use the measured weather information and performance data of
the first projectile to calibrate launches of subsequent
projectiles. For example, an interface of the handheld device 100
of FIG. 1 can display a recommended adjustment launch angle.
[0052] To launch the projectile 115, a pair of gas chambers 915 can
be engaged where the force of exiting gas causes the projectile 115
to launch. While shown as part of the projectile 115, the gas
chambers 915 may implement as part of the handheld device 100 of
FIG. 1. In one embodiment, the gas chambers 915 are filled with
compressed air. Causing the gas chambers 915 to engage can
constitute launch of the projectile 115. In one embodiment, gas
from the gas chamber is clean and odorless and causing the gas
chambers 915 to engage is silent. While a pair of gas chambers 915
are illustrated, it is to be appreciated by one of ordinary skill
in the art that the projectile 115 can implement with one gas
chamber, with more than two gas chambers, or with a different
launch implementation (e.g., a non-gas chamber launch
implementation, such as a catapult or use of a chemical rocket
propellant). In one embodiment, the projectile 115 can have the
pair of gas chambers 915 separate from the projectile 115 and this
separation causes the acquisition of images to begin.
[0053] FIG. 10 illustrates one embodiment of a system 1000 (e.g.,
configured for implementation on a handheld device described
herein) comprising an access component 1005 and an alignment
component 1010. The access component 1005 is configured to access a
stitched image 1015 of a location and an offline image 1020 of the
location. The alignment component 1010 is configured to align the
stitched image 1015 and the offline image 1020, where the stitched
image 1015 is a real-time image and the offline image 1020 is a
non-real-time image. The alignment component 1010 can produce an
aligned image 1025 as an output, where the aligned image 1025 is
the stitched image 1015 combined with the offline image 1020.
[0054] In one embodiment, the alignment component 1010 combines the
stitched image 1015 with the offline image 1020 through use of
feature extraction and feature mapping to form the aligned image
1025. The aligned image 1025 can be a new image or a modified
version of the stitched image 1015 or the offline image 1020. In
one example, the stitched image 1015 and the offline image 1020 can
be aligned with one another (to form the aligned image 1025) and
displayed together on an interface of the system 1000. These images
displayed together can provide a Warfighter with geospatial
information, blue force position, update of offline imagery and
maps, etc. With the aligned image 1025, geo-localization data
(e.g., map coordinates, latitude and longitude, etc.) can be
presented along with real-time information.
[0055] In one embodiment, the aligned image 1025 is presented on a
display of the handheld device 100. In one embodiment, the stitched
image 1015 is presented on the display of the handheld device 100
of FIG. 1 after the aligned image 1025 is created. When a part of
the stitched image 1015 is indicated by a user, geo-localization
data can be presented on the display that is derived from the
aligned image 1025 (e.g., latitude and longitude of a particular
position). In one embodiment, the stitched image 1015 is the
composite image and the offline image 1020 is the retained image.
In one embodiment, the composite image is the stitched image 1015
and the retained image is the offline image 1020.
[0056] In one embodiment, the stitched image is a compound image of
segment images. The segment images are of a lower resolution then a
resolution of the compound image. The offline image 1020 can be a
map of an area taken aerially. The map can be augmented with
information (e.g., latitude and longitude information, man-made
structure information, geographical landmark information, etc.).
The segment images can be images of the area taken from a
machine-launched projectile (e.g., the projectile 115 of FIG. 1
launched from the handheld device 100 of FIG. 1).
[0057] FIG. 11 illustrates one embodiment of a system 1100 (e.g.,
configured for implementation on a handheld device described
herein) comprising the access component 1005, the alignment
component 1010, a comparison component 1105, an analysis component
1110, and a feature component 1115. The comparison component 1105
is configured to make a comparison between the stitched image 1015
against the offline image 1020. The analysis component 1110 is
configured to perform an analysis on a result of the comparison.
The feature component 1115 is configured to find a common feature
set 1120 (e.g., one or more common feature) among the stitched
image 1015 and the offline image 1020 through employment of a
result of the analysis. The alignment component 1010 uses the
common feature set 1120 to align the stitched image 1015 and the
offline image 1020. The system 1100 can output the aligned image
1025, where the aligned image 1025 includes the common feature set
1120.
[0058] In one embodiment, a solider can cause the projectile 115 of
FIG. 1 to launch and capture the stitched image 1015 (e.g., capture
an initial image and stitch on the fly as subsequent images are
captured, capture individual images that are stitched together to
form the stitched image, etc.). The analysis component 1110 can
perform an evaluation of the stitched image 1015 and locate an
offline image 1020 that corresponds to the stitched image 1015
(e.g., perform a search for the offline image 1020 based on the
common feature set 1120 of the stitched image 1015 such that the
offline image 1020 has a feature set matching the common feature
set 1120). The comparison component 1105 can compare the stitched
image 1015 with the offline image 1020 and the analysis component
1110 can use a result of this comparison to determine if a found
image should be the offline image 1020. In one embodiment, a user
selects the offline image 1020.
[0059] The comparison component 1105 compares the stitched image
1015 against the offline image 1020. In one example, the comparison
component 1105 can identify a river in the stitched image 1015. The
comparison component 1105 can then find the river in the offline
image 1020. The analysis component 1110 determines that the rivers
match and based on this match, the feature component 1115
designates the river in the stitched image 1015 and the offline
image 1020 as part of the common feature set 1120. Using the common
feature set, the alignment component 1010 can align the stitched
image 1015 with the offline image 1020 to produce the aligned image
1025 (e.g., for Warfighter display and target destination
purposes). The alignment performed by the alignment component 1010
can be creating a new aligned image from the stitched image 1015
and the offline image, superimpose the stitched image 1015 on the
offline image 1020 to produce the aligned image 1025, superimpose
the offline image 1020 on the stitched image 1015 to produce the
aligned image 1025, etc. The system 1100 can cause the aligned
image 1025 to be displayed on a display (e.g., a display of a
device that retains at least part of the system 1100, a separate
display, etc.).
[0060] FIG. 12 illustrates one embodiment of a method 1200.
Receiving an indication of a location of a combination image can
occur at 1205 of the method 1200. The combination image results
from stitching a set of sub-combination images together, where the
combination image is of a higher resolution than a resolution of
individual images of the sub-combination image and where the
sub-combination images are a real-time image of a vicinity. At
1210, identifying a position of the location by comparing the
combination image against a non-real-time image of the vicinity
occurs. At 1215, causing an information set that is indicative of
the position to be disclosed on an interface occurs. In one
embodiment, the interface and the system 1200 of FIG. 12 are part
of the handheld device 100 of FIG. 1.
[0061] In one embodiment, the information set that is indicative of
the position is displayed concurrently with the combination image
on the interface. In one embodiment, the indication is received
from the interface (e.g., by a user touching an area of the
interface) and the set of sub-combination images are captured by a
launched projectile (e.g., the projectile 115 of FIG. 1).
[0062] In one embodiment, the information set is a command message
set (e.g., one or more command messages). An individual command of
the command message set causes a battle command action to occur,
when selected, at a locality that is indicated by the location of
the combination image. For example, an artillery soldier can use
the handheld device 100 of FIG. 1 and the handheld device 100 of
FIG. 1 can implement the method 1200. The artillery soldier can
designate a specific location of the combination image and request
an artillery strike at the specific location. The handheld device
100 can send an instruction to an artillery unit to cause the
artillery strike at the specific location. The action can be
sending the instruction, the artillery strike, etc. In one
embodiment, the action can be to designate locations for further
reconnaissance or situation updates or to cause update to occur
(e.g., instructions being retained in memory and then send to a
reconnaissance unit wirelessly).
[0063] FIG. 13 illustrates one embodiment of a system 1300
comprising a processor 1305 and a computer-readable medium 1310
(e.g., the memory). In one embodiment, the method 1200 of FIG. 12
may be implemented as computer executable instructions where the
computer-readable medium 1310 (e.g., Read Only Memory, Random
Access Memory, hard drive, etc.) stores the computer executable
instructions that when executed by a machine (e.g., the processor
1305) cause the machine to perform the method 1200.
[0064] Aspects disclosed herein allow an individual to perform
visual reconnaissance of his or her immediate surroundings. For
example, the individual can use aspects disclosed herein to capture
image of an area of radius of about one-hundred meters from the
individual. The handheld device 100 of FIG. 1 and the projectile
115 of FIG. 1 can be lightweight, portable, and relatively
unobtrusive. Deployment of the projectile 115 of FIG. 1 can be
difficult to detect by hostile elements (e.g., the projectile 115
can be shot about perpendicular to the ground). In one embodiment,
the artillery soldier can carry a case of projectiles 115 in his or
her pocket. After an initial projectile is used, it can be
separated from the handheld device 100 of FIG. 1 and a subsequent
projectile can be loaded and used. Due to the real-time nature of
an image captured by the projectile 115 of FIG. 1, a user can
quickly gain valuable information.
[0065] Various soldiers and other individuals can use aspects
disclosed herein to survey their immediate surroundings such that a
subject of surveillance is not made aware of the individual's
presence or presence of the surveillance. In one example,
dismounted soldiers can use aspects to surreptitiously reconnoiter
their immediate surroundings. In one example, police can use
aspects to track criminals, assist in crown control, gain
information in a hostage situation, etc. In one example, animal
inventory specialists or hunter can use aspects to scan for nearby
animals without alarming the animals. Aspects disclosed herein can
be used to avoid detection while gaining surveillance information.
Aspects disclosed herein can be used on manned or unmanned
vehicles.
[0066] The handheld device 400 of FIG. 4 can be a modified smart
phone with an attached device to receive data from the link 410 of
FIG. 4. Processing can be done on the handheld device 400 of FIG.
4. For example, the processor 1305 of FIG. 13 can be a processor of
the handheld device 400 of FIG. 4 and the processor 1305 of FIG. 13
can stitch the plurality of images together into the combination
image. The handheld device 400 of FIG. 4 can be hardened and
weatherproof to improve durability (e.g., be surrounded by a
durable plastic compound, have a cover molded in one piece to
minimize water entry, etc.).
[0067] To collect the plurality of images, the projectile 115 of
FIG. 9 can launch (e.g., upward towards the sky, underwater, etc.)
from the handheld device 400 of FIG. 4. The payload 810 of FIG. 9
can cause the camera or projectile 115 of FIG. 8 to pan around
viewing a somewhat large angle than the camera would not be able to
view without panning. The camera can aim underneath the projectile
115 of FIG. 9 and thus face downward. The camera can have a
relatively narrow capture angle (e.g., about 45 degrees) and the
plurality of images (e.g., two or more images) can be stitched
together to form a greater capture angle. For example, as shown in
FIG. 2, individual images 200-235 of FIG. 2 can each have a capture
angle of about 45 degrees, but when stitched together a combination
image can have a capture angle of about 360 degrees.
[0068] As the projectile 115 of FIG. 1 rises, the individual images
200-235 of FIG. 2 are captured. As a result, a first image (e.g.,
individual image 200 of FIG. 2) may be taken at a different
altitude than a second image (e.g., individual image 205 of FIG.
2). Due to the change in altitude, a discrepancy may exist between
individual image 200 of FIG. 2 and individual image 205 of FIG. 2,
such as different perspective, different distances of objects, etc.
The compensation component 505 of FIG. 5 can compensate for this
discrepancy. In one embodiment, an algorithm run by the
compensation component 505 of FIG. 5 can factor out wind or other
flight dynamic related displacement when the handheld device 500 of
FIG. 5 creates the combination image. In addition, the handheld
device 500 of FIG. 5 can use other components, such as an
accelerometer of the projectile 115 of FIG. 1 to collect speed and
direction vectors of individual images 200-235 of FIG. 2, to
determine the discrepancy and determine an appropriate corrective
action.
[0069] In one embodiment, the projectile 115 of FIG. 1 captures a
relatively large number of images. The image stitch component 110
of FIG. 1 uses an image size determination algorithm to determine a
phase of an initial payload descent. Based on this phase, the image
stitch component 110 of FIG. 1 can select the plurality of images
used to produce the composite image from the relatively large
number of images. In one embodiment, a timer can be employed to
select which images are part of the plurality of images and/or when
the begin/end image capturing by the projectile 115 of FIG. 1.
[0070] The handheld device 100 of FIG. 1 can include a launch
chamber where the projectile 115 of FIG. 1 is launched from and
where a user loads the projectile 115 of FIG. 1. The launch chamber
can include a shaft or barrel with screw guides that facilitate the
projectile 115 of FIG. 1 to launch with a spiral. In one
embodiment, the projectile 115 of FIG. 9 can position or otherwise
use the gas chambers 915 to cause launch with a spiral. Spiraling
of the projectile 115 of FIG. 9 can be used to capture the
individual images 200-235 of FIG. 2 at different angles (e.g., thus
the camera remains static with reference to the projectile 115 of
FIG. 9). In one embodiment, at least one fin (e.g., retractable,
fixed, etc.) of the housing 905 of FIG. 9 can cause spiraling of
the projectile 115 of FIG. 9 (e.g., the projectile 115 of FIG. 9
being of a size of a cigarette or an average human male
finger).
[0071] It is to be appreciated by one of ordinary skill in the art
that references to one item in one figure is not limiting to that
embodiment. For example, a reference to the handheld device 100 of
FIG. 1 can also be applied to the handheld device 400 of FIG.
4.
* * * * *