ExifTool Forum

ExifTool => Newbies => Topic started by: SM26 on February 13, 2019, 09:41:18 AM

Title: Extracting GPS data from INSV file
Post by: SM26 on February 13, 2019, 09:41:18 AM
Hello everyone

I have a problem, and I hope you can direct me.

I have a 360 camera, Insta360 One X.
It produce an unstitched file in INSV (proprietary) file format.
I can change it to MP4 and it works just fine
or I can use a dedicated programme to produce a stitched version in MP4 format.

The camera records GPS data, and when I use the Android app with the INSV files, I can see all of the GPS data

I was wondering if it was possible to extract all of the GPS data (preferably in GPX format) out of it.

My system:
Windows 10, 64 Bit.
ExifTool V11.26
ExifToolGUI V5.16

In the Gui, I've clicked "Export data to XMP"
but I get "LargeFileSupport not enabled" warning.
I've made a .ExifTool_config file in the same directory as the exiftool itself, with an additional 2 lines (LargeFileSupport => 1 & print "LOADED!\n"; ):

%Image::ExifTool::UserDefined::Options = (
    CoordFormat =&gt; '%.6f',  <span class=com># change default GPS coordinate format</span>
    Duplicates =&gt; 1,        <span class=com># make -a default for the exiftool app</span>
    GeoMaxHDOP =&gt; 4,        <span class=com># ignore GPS fixes with HDOP &gt; 4</span>
    RequestAll =&gt; 3,        <span class=com># request additional tags not normally generated</span>
    LargeFileSupport => 1,

);
print "LOADED!\n";
<span class=com>#------------------------------------------------------------------------------</span>
1;  <span class=com>#end</span>
</pre>
<hr>
<p class='lf'># <a href="index.html">&lt;-- Back to ExifTool home page</a></p>
</body>
</html>

I still get the warning, and I've read FAQ 11 and followed it to the letter, still happening.

Edit:
If it matters, I've also tried to use MetaData++, which is based on ExifTool. still no luck.

I would really appreciate any direction you could throw at me.
I'm also currently upload the INSV file to Google Drive so you can take a look for yourself (it's gonna take some time, it's a 5.8GB file).

TIA
-SM26
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 14, 2019, 08:11:18 AM
Here is the INSV file if someone is willing to take a look:
https://drive.google.com/open?id=1bsEVAGx4Co1DqINziaXF2hy-HinD54kl
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 14, 2019, 09:55:32 AM
From the command line I get the following information from your file with -api largefilesupport added to the command:

Code: [Select]
> exiftool tmp/VID_20190126_125708_00_046.insv -api largefilesupport
ExifTool Version Number         : 11.26
File Name                       : VID_20190126_125708_00_046.insv
Directory                       : tmp
File Size                       : 5.9 GB
File Modification Date/Time     : 2019:02:14 09:44:40-05:00
File Access Date/Time           : 2019:02:14 09:45:58-05:00
File Inode Change Date/Time     : 2019:02:14 09:44:40-05:00
File Permissions                : rw-r--r--
File Type                       : MP4
File Type Extension             : mp4
MIME Type                       : video/mp4
Major Brand                     : MP4 Base w/ AVC ext [ISO 14496-12:2005]
Minor Version                   : 0.0.0
Compatible Brands               : avc1, isom
Movie Data Size                 : 6271586288
Movie Data Offset               : 48
Movie Header Version            : 0
Create Date                     : 2019:01:26 12:56:58
Modify Date                     : 2019:01:26 12:56:58
Time Scale                      : 1000
Duration                        : 0:09:49
Preferred Rate                  : 1
Preferred Volume                : 100.00%
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Preview Time                    : 0 s
Preview Duration                : 0 s
Poster Time                     : 0 s
Selection Time                  : 0 s
Selection Duration              : 0 s
Current Time                    : 0 s
Next Track ID                   : 4
Track Header Version            : 0
Track Create Date               : 2019:01:26 12:56:58
Track Modify Date               : 2019:01:26 12:56:58
Track ID                        : 1
Track Duration                  : 0:09:49
Track Layer                     : 0
Track Volume                    : 0.00%
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Image Width                     : 3840
Image Height                    : 1920
Media Header Version            : 0
Media Create Date               : 2019:01:26 12:56:58
Media Modify Date               : 2019:01:26 12:56:58
Media Time Scale                : 50000
Media Duration                  : 0:09:49
Handler Type                    : Video Track
Handler Description             : Ambarella AVC
Graphics Mode                   : srcCopy
Op Color                        : 0 0 0
Compressor ID                   : avc1
Source Image Width              : 3840
Source Image Height             : 1920
X Resolution                    : 72
Y Resolution                    : 72
Compressor Name                 : Ambarella AVC encoder
Bit Depth                       : 24
Pixel Aspect Ratio              : 1:1
Stereoscopic 3D                 : Monoscopic
Metadata Source                 : .AMBA Spherical Video
Pose Yaw Degrees                : 0
Pose Pitch Degrees              : 0
Pose Roll Degrees               : 0
Projection Bounds Top           : 0
Projection Bounds Bottom        : 0
Projection Bounds Left          : 0
Projection Bounds Right         : 0
Video Frame Rate                : 50
Spherical                       : true
Stitched                        : true
Stitching Software              : AMBA Spherical Video
Projection Type                 : equirectangular
Stereo Mode                     : mono
Source Count                    : 2
Track Header Version            : 0
Track Create Date               : 2019:01:26 12:56:58
Track Modify Date               : 2019:01:26 12:56:58
Track ID                        : 2
Track Duration                  : 0:09:49
Track Layer                     : 0
Track Volume                    : 100.00%
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 2019:01:26 12:56:58
Media Modify Date               : 2019:01:26 12:56:58
Media Time Scale                : 48000
Media Duration                  : 0:09:49
Handler Type                    : Audio Track
Handler Description             : Ambarella AAC
Balance                         : 0
Audio Format                    : mp4a
Audio Channels                  : 2
Audio Bits Per Sample           : 0
Audio Sample Rate               : 48000
Track Header Version            : 0
Track Create Date               : 2019:01:26 12:56:58
Track Modify Date               : 2019:01:26 12:56:58
Track ID                        : 3
Track Duration                  : 0:09:49
Track Layer                     : 65535
Track Volume                    : 0.00%
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 2019:01:26 12:56:58
Media Modify Date               : 2019:01:26 12:56:58
Media Time Scale                : 1000
Media Duration                  : 0:09:49
Handler Type                    : NRT Metadata
Handler Description             : Ambarella EXT
Meta Format                     : camm
Warning                         : [minor] The ExtractEmbedded option may find more tags in the movie data
Avg Bitrate                     : 85.1 Mbps
Base Name                       : vid
File Extension                  : insv
File Type Description           : MPEG-4 video
Image Size                      : 3840x1920
Megapixels                      : 7.4
Rotation                        : 0

So this should work for you if you can get LargeFileSupport enabled.

The config file you posted contains some HTML markup which would stop it from working.  Try this config file named ".ExifTool_config" in the exiftool.exe directory and/or your home directory:

Code: [Select]
%Image::ExifTool::UserDefined::Options = (
    LargeFileSupport => 1,
);
1; #end

- Phil

Edit:  But wait.  You wanted to extract GPS data.  There is no valid GPS that I can find in this file.  It does contain some GPS information, but it is all void in this video (ie. the receiver didn't acquire a GPS fix).
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 18, 2019, 04:10:53 AM
Thank you Phil,

I've tried this short config file you posted and it works!

and now to the main event, GPS Data.
I can try to create some new testing materials, this time short in duration and small in size for ease of processing.
Do you think that this might help?

I know for a fact that via the app, I can post pictures to Google Street View, so I'm guessing there must be some GPS data hidden somewhere....
I've also dug in their app, nothing useful that I can see.

I'll post here a link to Google Drive when I'll finish uploading all of the new test data.

Thanks Phil!

- Or
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 18, 2019, 07:38:12 AM
I'll take a look at the files that you upload.  Please confirm that the app can obtain GPS from this file alone (often cameras write sidecar metadata files.  Be sure it isn't getting the information from here.  Also, let me know the GPS coordinates it obtains, and whether it gives a single GPS point or a full track from the video.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 19, 2019, 10:28:23 AM
Ok, so I've done 3 short test videos.

But before that, here is how I set it up:

Camera is Insta360 One X, FW V1.17.16.
4K 30 FPS, with "stats" indicator turns on in the app, with "Google Street View" indicator turns on in the app.
Insta360 say that the GPS data is recorded from the phone's GPS, but from what I can tell, it's not inside the app data.

1. Shot first video (vid_050.insv) , saw that I can see GPS data in the app.
2. Backup the INSV file to the PC
3. Format phone, switched to android 9.
4. Load back the INSV file into the phone, GPS data is in the app.
5. Shot while the phone is stationary, and I move around with the camera (Vid_051.insv).
6. Shot while the camera is stationary, and I move around with the phone (Vid_052.insv).

I've also tried to use:
Code: [Select]
ffprobe vid_050.insv output:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'vid_050.insv':
  Metadata:
    major_brand     : avc1
    minor_version   : 0
    compatible_brands: avc1isom
    creation_time   : 2019-02-19T16:26:32.000000Z
  Duration: 00:00:58.49, start: 0.000000, bitrate: 46167 kb/s
    Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, smpte170m), 3840x1920 [SAR 1:1 DAR 2:1], 44678 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      creation_time   : 2019-02-19T16:26:32.000000Z
      handler_name    :  Ambarella AVC
      encoder         : Ambarella AVC encoder
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2019-02-19T16:26:32.000000Z
      handler_name    :  Ambarella AAC
    Stream #0:2(eng): Data: none (camm / 0x6D6D6163), 7 kb/s (default)
    Metadata:
      creation_time   : 2019-02-19T16:26:32.000000Z
      handler_name    :  Ambarella EXT
Unsupported codec with id 0 for input stream 2

Followed by:
Code: [Select]
ffmpeg -y -i vid_050.insv -codec copy -map 0:2 -f rawvideo out-050.binwhich basically gave me a whole lot of:
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 50650 
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 50750 
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 50850 
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 50950 
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 51050 


XMP output is:
Code: [Select]
<?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?>
<x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='Image::ExifTool 11.26'>
<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>

 <rdf:Description rdf:about=''
  xmlns:GPano='http://ns.google.com/photos/1.0/panorama/'>
  <GPano:ProjectionType>equirectangular</GPano:ProjectionType>
  <GPano:StitchingSoftware>AMBA Spherical Video</GPano:StitchingSoftware>
 </rdf:Description>

 <rdf:Description rdf:about=''
  xmlns:GSpherical='http://ns.google.com/videos/1.0/spherical/'>
  <GSpherical:SourceCount>2</GSpherical:SourceCount>
  <GSpherical:Spherical>True</GSpherical:Spherical>
  <GSpherical:StereoMode>mono</GSpherical:StereoMode>
  <GSpherical:Stitched>True</GSpherical:Stitched>
 </rdf:Description>
</rdf:RDF>
</x:xmpmeta>
<?xpacket end='w'?>

Did this for 051 and 052 with no change.
Files can be seen here:
https://drive.google.com/open?id=1NbzpGAF-mSrpNAzMDuyDGfF0_SV9_92S

Thanks in advance,
- Or
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 19, 2019, 10:49:09 AM
OK, I think I can see where the GPS is stored.  Can you tell me ALL of the information displayed by "the app" for the first fix of VID_050?  I need the GPS coordinates, speed, time and whatever else is extracted to be able to decode this from the binary data.

- Phil

Edit:  What is "the app" BTW?  I could run it myself it there is a Mac version available.
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 19, 2019, 11:03:05 AM
from vid_050.insv, the data in the app is:

Time: 00:28 (I can't see frame number in the app)
location: 32° 5′ 16″ N 34° 51′ 22″ E
direction: 288° W
Route +  a dot where I'm on the route + number of Km from the start.
Slope: 47.7%
Elevation: 61.63M
Speed: 2.2 Km/h

I've added a screen shot from the app inside the google drive folder.
(I know that the picture is all black, it's fine, the camera was in my pocket)
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 19, 2019, 12:31:51 PM
Thanks.  Decoding all this will take some time.  I have found 4 different types of unknown records.  One of these records contains the following information, which gives time/lat/lon plus some other numbers that I don't yet understand.  Here is the dump of the start of this record as decoded from VID_050:

Code: [Select]
Tue Feb 19 09:26:32 2019 0 0 A 32.08812118 N 34.85583192 E 0 0 0 0 0 3.38119339942932
Tue Feb 19 09:26:32 2019 0 0 A 32.08812118 N 34.85583192 E 0 0 0 0 0 3.38119339942932
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
...

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 20, 2019, 09:17:18 AM
More information:  Here is a dump of the first ten lines of each of the 5 different unknown record types I can find in VID_050.  Please let me know if you know the significance of any of the floating-point numbers in records 0x300, 0x400 or 0x700 (other than the lat/lon in record 0x700, which is obvious).  The first number in records 0x300 and 0x400 is a time code in milliseconds, but I don't know how to relate this to either clock time or a frame in the video.  Records 0x101 and 0x200 are presented as a hex dump, while the others are decoded into variable types that seemed appropriate.

Also, is "IXE3518NYGDB3N" the serial number of your Insta360 One2?

Code: [Select]
============ Record 0x101
    0000: 0a 0e 49 58 45 33 35 31 38 4e 59 47 44 42 33 4e [..IXE3518NYGDB3N]
    0010: 12 0d 49 6e 73 74 61 33 36 30 20 4f 6e 65 32 1a [..Insta360 One2.]
    0020: 0f 76 31 2e 31 37 2e 31 36 5f 62 75 69 6c 64 31 [.v1.17.16_build1]
    0030: 2a 6e 32 5f 31 34 37 37 2e 39 38 38 5f 31 35 32 [*n2_1477.988_152]
    0040: 37 2e 39 31 35 5f 31 35 32 33 2e 37 34 36 5f 30 [7.915_1523.746_0]
    0050: 2e 30 30 30 5f 30 2e 30 30 30 5f 30 2e 30 30 30 [.000_0.000_0.000]
    0060: 5f 31 34 38 31 2e 30 37 38 5f 34 35 34 34 2e 33 [_1481.078_4544.3]
    0070: 31 32 5f 31 35 32 34 2e 35 38 33 5f 30 2e 31 31 [12_1524.583_0.11]
    0080: 39 5f 2d 30 2e 31 36 35 5f 2d 31 37 39 2e 37 30 [9_-0.165_-179.70]
    0090: 31 5f 36 30 38 30 5f 33 30 34 30 5f 32 33 32 33 [1_6080_3040_2323]
[...]
============ Record 0x200
    0000: 00 00 00 01 7a 01 01 00 00 23 e9 10 00 3d 4c 04 [....z....#...=L.]
    0010: 00 02 00 01 20 10 00 00 00 01 67 4d 40 33 9a 64 [.... .....gM@3.d]
    0020: 01 e0 07 8d 80 b7 06 06 06 40 00 00 fa 40 00 3a [.........@...@.:]
    0030: 98 3a 18 00 0b eb c0 00 02 fa f0 ae f2 e3 43 00 [.:............C.]
    0040: 01 7d 78 00 00 5f 5e 15 de 5c 3e 11 08 b7 00 00 [.}x.._^..\>.....]
    0050: 00 01 68 ee 38 80 00 00 00 01 65 b8 01 00 00 8f [..h.8.....e.....]
    0060: f5 2a 11 14 72 19 cc 78 2f e9 1f 26 ea fc 1c 79 [.*..r..x/..&...y]
    0070: 44 64 6a 67 a3 7a 94 d1 3c 50 52 93 8a ab 2c ec [Ddjg.z..<PR...,.]
    0080: d1 10 19 c3 9d 13 44 d8 08 2f 6c a8 30 12 60 9c [......D../l.0.`.]
    0090: b3 47 2e 04 21 22 0c 32 5b b3 4f 77 37 69 91 1f [.G..!".2[.Ow7i..]
[...]
============ Record 0x300
50.394 0.555908203125 -0.826416015625 0.10595703125 -0.00426105774412678 0.00319579330809508 0.00319579330809509
50.396 0.558349609375 -0.828369140625 0.10205078125 -0.00213052887206339 0.00213052887206339 0.00213052887206339
50.398 0.55712890625 -0.830322265625 0.103759765625 -0.00426105774412678 0.00106526443603169 0.00213052887206339
50.400 0.55908203125 -0.8291015625 0.105224609375 -0.00213052887206339 0 0.00106526443603169
50.402 0.560546875 -0.8291015625 0.10205078125 -0.00213052887206339 0 0.00106526443603169
50.404 0.55859375 -0.830322265625 0.101318359375 -0.00213052887206339 0.00106526443603169 0
50.406 0.559814453125 -0.82958984375 0.103271484375 -0.00213052887206339 0.00213052887206339 0
50.408 0.5595703125 -0.828857421875 0.10595703125 -0.00213052887206339 0.00426105774412678 -0.00106526443603169
50.410 0.561279296875 -0.827880859375 0.10791015625 -0.00213052887206339 0.00426105774412678 -0.00319579330809509
50.412 0.558349609375 -0.830322265625 0.107177734375 -0.00213052887206339 0.00319579330809508 -0.00213052887206339
[...]
============ Record 0x400
50.391 0 0 0 96 1.25778567790985
50.425 0 0 0 96 1.25778567790985
50.458 0 0 0 96 1.25778567790985
50.492 0 0 0 96 1.25778567790985
50.525 0 0 0 96 1.25778567790985
50.558 0 0 0 96 1.25778567790985
50.592 0 0 0 96 1.25778567790985
50.625 0 0 0 96 1.25778567790985
50.658 0 0 0 96 1.25778567790985
50.692 0 0 0 96 1.25778567790985
[...]
============ Record 0x700
Tue Feb 19 09:26:32 2019 0 0 A 32.08812118 N 34.85583192 E 0 0 0 0 0 3.38119339942932
Tue Feb 19 09:26:32 2019 0 0 A 32.08812118 N 34.85583192 E 0 0 0 0 0 3.38119339942932
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
[...]

It seems that Record 0x300 may contain accelerometer information from the phone because it contains more interesting numbers for VID_052:

Code: [Select]
============ Record 0x300
120.810 0.697021484375 0.349609375 -0.662353515625 0.181094954125388 -0.385625725843473 -0.0372842552611093
120.814 0.701171875 0.355224609375 -0.663330078125 0.196008656229832 -0.382429932535378 -0.0362189908250776
120.816 0.70068359375 0.35986328125 -0.662353515625 0.208791829462212 -0.380299403663315 -0.0351537263890459
120.818 0.69970703125 0.367919921875 -0.664306640625 0.219444473822529 -0.376038345919188 -0.0340884619530142
120.820 0.696533203125 0.368408203125 -0.659912109375 0.227966589310783 -0.371777288175061 -0.0319579330809508
120.822 0.69580078125 0.3740234375 -0.664794921875 0.232227647054909 -0.365385701558871 -0.0276968753368241
120.824 0.69482421875 0.375 -0.6572265625 0.234358175926973 -0.360059379378713 -0.0223705531566656
120.826 0.693359375 0.3720703125 -0.654541015625 0.234358175926973 -0.356863586070618 -0.0191747598485705
120.828 0.692626953125 0.372314453125 -0.654052734375 0.234358175926973 -0.352602528326491 -0.013848437668412
120.830 0.69580078125 0.3671875 -0.655029296875 0.234358175926973 -0.350471999454427 -0.00852211548825355
[...]

- Phil

Edit: I think it is likely that after the time code, the next 3 numbers are XYZ acceleration in units of g, and the last 3 numbers are angular velocities about each axis (probably rad/s).
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 21, 2019, 07:48:06 AM
I've figured out a bit more, and can now decode GPSSpeed, GPSTrack and GPSAltitude, but I need to know what units the GPSSpeed is measured in.  For VID_051, the GPSSpeed starts with a value of 0.83.  I'm thinking this may be in m/s, which would be 2.99 km/h.  What does your app show for the speed at the start of this video?

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 21, 2019, 12:08:54 PM
Hello Phil,

First of all, I want to say a huge thank you for taking the time and handling it.
I'll try to answer everything as best of my knowledge, let's hope I don't forget anything.

1. From the dump of Vid_050:
Code: [Select]
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
in the first line, I'm guessing that the info we have here is: longitude, latitude, elevation.
on the second line, I have no idea what are the rest of this stuff: 1.62999999523163 64 3.67499995231628

2. As far as I know, accuracy of longitude and latitude data is measured up to 7 numbers after the floating point.
here I can count 8, which means that it's accurate to about 1 meter, 7 is fine by me (accurate to about 10 meters)
for me accuracy of elevation data of 4 numbers after the floating point is also sufficient.
I don't mind the extra accuracy, but just to let you know what I consider "accurate data measurement"

3. Regarding time code: I know that the video is 30 FPS, is this helping?
I don't think that the phone is sampling the GPS data every second, but I might be mistaking.

4. Regarding 0x101:
Yes, IXE3518NYGDB3N is the camera S/N.
Insta360 is the name of the company
One2 is the technical name of the One X (just like phones, where Samsung Galaxy S5 is SM-G900F)
V1.17.16 is my firmware version.
lines 30 and onward - I have no idea.

5. Record 0x200 also makes no sense to me.

6. Just like you said, 0x300 seems like 6 axis data to me.
I have no idea how to interpret it (what axis is what number) but I'm assuming it's in the following order:
Roll: rotation around the front-to-back axis, Pitch: rotation around the side-to-side axis, Yaw: rotation around the vertical axis
And then the next 3 numbers are from the accelerometer.
I have a friend who might be able to help a bit, I'll run it by him.

7. Regarding recorded 0x400, can it be that 96 is Fahrenheit of the phone? does it fluctuate much?

8. When looking at Vid_050 dump:
Code: [Select]
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Can it be that the two 0 before the "A" is altitude? and that's why it's 0? (don't know why you got 0 twice though)
And after the "E", 224 looks like a direction to me, like SW on a compass.

9. Vid_051 is when the phone is stationary and the camera moves.
So, logicly, the speed should be 0
(because the base assumption is that the camera don't get a GPS signal, only the phone)
I've checked the video, the data I see in the app for Vid_052 at 00:00 is:
speed: 0.58 km/h
elevation: 58.33
slope: 0.0%
routh: 0.00km
direction: 293° NW
location: 32°5'16"N , 34°51'22"E

And at 00:03 (approx when I put my phone down and start walking around with the camera):
speed: 0.00 km/h
elevation: 58.33
slope: 29.7%
routh: 0.00km
direction: 0° N
location: 32°5'16"N , 34°51'22"E

I found out in the app that for Vid_050 at 00:15 I get to speed of 2.99 km/h, here is the full data:
speed: 2.99 km/h
elevation: 89.96M
slope: 100.0%
routh: 0.00km
direction: 86° E
location: 32°5'17"N , 34°51'21"E



Sorry this post got so long,
for me all of this is really an uncharted territory and this is the first time I'm dealing with this kind of a challenge
Thanks again
-Or
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 21, 2019, 12:37:26 PM
Great, thanks.  I'm pretty confident now that I'm extracting GPSLatitude/Longitude/Speed/Altitude/Track correctly.  Note that you must use the -ee option to extract this information.

I've just released ExifTool 11.28 with this update.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 21, 2019, 12:49:16 PM
Great!

Thanks,
I'll download the new version
and create some more test samples, we will see how it goes from there.
(so far I've only used the GUI version, so I'll probably need a day or two to read the documentation about the CMD version.)
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 06:45:00 AM
Great, thanks.  I'm pretty confident now that I'm extracting GPSLatitude/Longitude/Speed/Altitude/Track correctly.  Note that you must use the -ee option to extract this information.

I've just released ExifTool 11.28 with this update.

- Phil

Hello Phil,

I think I got it,
one thing I'm not 100% sure about,
I use the command: exiftool -ee -p gpx.fmt Vid_052.insv > Vid052.gpx

And I've used the GPX.FMT file from here:
https://www.sno.phy.queensu.ca/~phil/exiftool/geotag.html#GPX

currently I see: longitude, latitude, elevation.
Should I modify the FMT file somewhat?
just want to make sure I'm not missing out on any data available.

Just found this little gem:
http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,5095.msg46514.html#msg46514
do we have some gyro data to add to the FMT file like so; "ACC $framenumber $accelerometer" ?

Thanks in advance.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 25, 2019, 07:03:27 AM
Yes.  Modify the gps.fmt file to write anything you want.  You may have to consult the GPX specification to see how to format this properly.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 07:45:26 AM
Sorry, But I don't quite follow

Like we said earlier, record 0x300 seems like accelerometer information.

When I look at the GoPro and Ricoh tag pages, I see they have accelerometer data.
What and how should I alter the FMT to show all of the data?

I've made small changes to the FMT file, but now it shows up blank...

#------------------------------------------------------------------------------
# File:         gpx.fmt
#
# Description:  Example ExifTool print format file for generating GPX track log
#
# Usage:        exiftool -p gpx.fmt FILE [...] > out.gpx
#
# Requires:     ExifTool version 10.49 or later
#
# Revisions:    2010/02/05 - P. Harvey created
#               2018/01/03 - PH Added IF to be sure position exists
#               2018/01/06 - PH Use DateFmt function instead of -d option
#
# Notes:     1) Input files must contain GPSLatitude and GPSLongitude.
#            2) Add the -ee option to extract the full track from video files.
#            3) The -fileOrder option may be used to control the order of the
#               generated track points when processing multiple files.
#------------------------------------------------------------------------------
#[HEAD]<?xml version="1.0" encoding="utf-8"?>
#[HEAD]<gpx version="1.0"
#[HEAD] creator="ExifTool $ExifToolVersion"
#[HEAD] xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
#[HEAD] xmlns="http://www.topografix.com/GPX/1/0"
#[HEAD] xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">
#[HEAD]<trk>
#[HEAD]<number>1</number>
#[HEAD]<trkseg>
#[IF]  $gpslatitude $gpslongitude
#[IF]  $accelerometer
#[BODY]<trkpt lat="$gpslatitude#" lon="$gpslongitude#">
#[BODY]  <ele>$gpsaltitude#</ele>
#[BODY]  <acc>$accelerometer#</acc>
#[BODY]  <time>${gpsdatetime#;DateFmt("%Y-%m-%dT%H:%M:%SZ")}</time>
#[BODY]</trkpt>
#[TAIL]</trkseg>
#[TAIL]</trk>
#[TAIL]</gpx>

From the GoPro page, I think that the following tags are interesting:
ACCL, GLPI, GPRI, GPS5, GYRO, MAGN
And from the Ricoh page, I think that the following tags are interesting:
Ricoh ThetaSubdir Tags ->0x0003 Accelerometer, 0x0004 Compass

Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 25, 2019, 07:57:06 AM
The main problem with what you are trying to do is that the accelerometer readings aren't timed the same as the GPS, so they can't be combined like this.  They are based on a TimeCode, while the GPS readings are based on GPSDateTime.  You can add the GPSSpeed and GPSTrack to the GPS records in the GPX file, but it would take some work to be able to relate the TimeCode values to a clock time so you could add in the accelerometer information.  Frankly, I don't know how to do this.  Use this command to see all of the available information:

exiftool -ee -G3 FILE

In this output, each document number represents the information extracted for a specific SampleTime, TimeCode or GPSDateTime.  Note that GPS is sampled at a much different rate than the accelerometer readings.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 08:53:06 AM
Ok, I think I have an idea,

Based on Garmin data that I found (and let's hope that Insta360 log data the same way as Garmin)
each GPS record correspond with 10 Accelerometer and Angular Velocity docs. (for now let's leave the offset out of this)
https://www8.garmin.com/xmlschemas/AccelerationExtensionv1.xsd
Let's hope that I what the XYZ value match correspondingly

So, for file 052, the EOF should look like this:
 <trkpt lat="32.08802777777778" lon="34.85615555555556">
        <ele>61.5087</ele>
        <time>2019-02-19T14:29:02Z</time>
        <extensions>
          <acc:AccelerationExtension xmlns:acc="http://www.garmin.com/xmlschemas/AccelerationExtension/v1">
            <acc:accel x="0.065185546875" y="-0.977294921875" z="0.24462890625" />
            <acc:accel x="0.06591796875" y="-0.97607421875" z="0.242431640625" />
            <acc:accel x="0.064453125" y="-0.975830078125" z="0.23974609375" />
            <acc:accel x="0.064453125" y="-0.975341796875" z="0.24267578125" />
            <acc:accel x="0.0654296875" y="-0.979248046875" z="0.2421875" />
            <acc:accel x="0.061279296875" y="-0.977294921875" z="0.239501953125" />
            <acc:accel x="0.065185546875" y="-0.977294921875" z="0.24462890625" />
            <acc:accel x="0.0625" y="-0.9765625" z="0.24267578125" />
            <acc:accel x="0.0625" y="-0.97265625" z="0.239501953125" />
            <acc:accel x="0.066162109375" y="-0.974609375" z="0.240478515625" />
          </acc:AccelerationExtension>
          <Gyro:Angular Velocity>
            <acc:accel x="0.00639158661619017" y="0.0149137021044437" z="0.00852211548825355" />
            <acc:accel x="0.00852211548825355" y="0.0149137021044437" z="0.00852211548825355" />
            <acc:accel x="0.00639158661619017" y="0.0149137021044437" z="0.00852211548825355" />
            <acc:accel x="0.00852211548825355" y="0.0149137021044437" z="0.00958737992428525" />
            <acc:accel x="0.00852211548825355" y="0.0159789665404754" z="0.0106526443603169" />
            <acc:accel x="0.00852211548825355" y="0.013848437668412" z="0.00852211548825355" />
            <acc:accel x="0.00639158661619017" y="0.013848437668412" z="0.00745685105222186" />
            <acc:accel x="0.00639158661619017" y="0.013848437668412" z="0.00852211548825355" />
            <acc:accel x="0.00639158661619017" y="0.0149137021044437" z="0.00852211548825355" />
            <acc:accel x="0.00639158661619017" y="0.0127831732323803" z="0.00958737992428525" />
          </Gyro:Angular Velocity>
        </extensions>
      </trkpt>


Any change to make something like this?
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 25, 2019, 09:03:36 AM
I still don't know how to relate the accelerometer TimeCode values to the GPSDateTime.  You can't just assume that the first TimeCode corresponds with the first GPSDateTime because.

1. It is quite likely that the GPS system takes a while to acquire a lock, and hence the first reading may be a while after the start of the video.

2. You would likely be wrong by half of the GPS sampling period on average even if they did start at the same time.

But even if you can relate these times, constructing the combined GPX output file is currently beyond the capabilities of ExifTool, and would require some dedicated scripting to achieve.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 09:30:10 AM
Ok,
I'll spend some more time with it, maybe I'll manage to learn something new.

Last question (hopefully)
I have a file that is 18 GB (19,220,410,368 bytes to be exact)

When I run Exiftool -ee -p gpx2.fmt Vid_long.insv > Vid_Long.gpx
I get:
Code: [Select]
Out of memory!
for the record, my config file has "LargeFileSupport => 1" in it.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 25, 2019, 09:34:07 AM
Unfortunately there is a limit on the amount of memory that a Windows command-line process may allocate.  I don't know why this limit is imposed, but it is around 300 MB or so.  Mac and Linux have no such limitation.  I don't know any way around this on Windows.  Processing very large video files with the -ee option does use a fair bit of memory.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 09:37:09 AM
oh, Ok.
Thanks!

good think I have a linux laptop laying around
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 28, 2019, 04:28:18 AM
So, just to sum it up
Phil, you have helped me a lot.

Insta360 replied to my email, and they said that they are not going to help understand the Gyro data.

But nevertheless, for your time and effort
as a token of my appreciation, let me buy you a pint!

PayPal's Transaction ID: 3RE879167D940642R
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 28, 2019, 07:00:32 AM
Thanks!

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on September 30, 2019, 11:33:00 AM
Hello Everyone,

I am trying to use Exiftool 11.66 for extracting GPS data from .insv file taken using (Insta 360 ONEX camera).

The command I used:
Code: [Select]
exiftool -ee -p "GPS $gpslatitude $gpslongitude" -n "C:\VID_003.insv" > VID_003.txt
I am getting results. Latitude looks right however Longitude shows incorrect value.

GPS 40.74103544 -5450.57354413288

Appreciated your help.

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on September 30, 2019, 11:38:42 AM
Hi Yamuna,

It could be that the format of this metadata is somehow different than my current INSV sample.

Could you possibly send me a (hopefully small) INSV file with GPS information?  I'll see if I can figure out what is going on.  My email is phil at owl.phy.queensu.ca

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on September 30, 2019, 01:00:12 PM
Hi Phil,

Thanks for the swift response. sent you the sample insv file.

Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on September 30, 2019, 01:20:01 PM
Hi Yamuna,

I got the sample, thanks.

This was a bug in ExifTool that I have fixed in version 11.67 (just released).

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on September 30, 2019, 01:51:04 PM
Thanks Phil, It works.
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 04, 2019, 12:30:19 PM
Hi Phil,

When I try to extract larger file (9GB) I am getting "out of Memory" error.

exiftool -ee -p "$gpsdatetime $gpslatitude $gpslongitude" -n "test.insv"> test.csv

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 04, 2019, 12:32:43 PM
Could you attach the output of the -v3 command for this file?
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 04, 2019, 01:56:25 PM
Attached the image file.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 04, 2019, 01:57:32 PM
That doesn't help as I only see the start of the output.  Also, you will need to add the LargeFileSupport option:

So try running this command and attaching the "out.txt" file (or email it to me if you want: phil at owl.phy.queensu.ca)

exiftool -api largefilesupport -v3 FILE > out.txt

Thanks.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 04, 2019, 02:07:09 PM
Attached. Thanks.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 07, 2019, 07:21:39 AM
Unfortunately I can't figure out the problem from the verbose output.  I will need the sample file to be able to reproduce the problem.  Is there any way you could upload this to a file sharing service somewhere?  Then email me a link (phil at owl.phy.queensu.ca)

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 07, 2019, 11:30:52 AM
Hi Phil,

I sent you the video link to download 9gb insv file.
Let me know once you download the file, i will remove the link.

Thanks,
yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 07, 2019, 12:12:05 PM
Hi Yamuna,

I got the file, thanks.

The problem is simply that the file contains too much metadata to read into the limited memory space available to the Windows exe version of ExifTool (about 200 MB or so).  The file contains about 800000 accelerometer records, which is the main problem.  I'll see about limiting the number of these that ExifTool will read to work around this limitation.  It contains about 16500 GPS records, but ExifTool should be able to read all of these.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 07, 2019, 02:07:04 PM
Hi Phil,

I tried from Linux Virtual machine and I got into the same issue.

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 07, 2019, 08:57:01 PM
I'm surprised.  At any rate, ExifTool 11.70 will fix this by not attempting to load all of the accelerometer data unless you use the -m option.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 09:42:40 AM
Thanks Phil.
When are you planning to release version 11.70 ?

Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 09:47:10 AM
Probably within a week.  I'm attaching the updated code which you can use to replace lib/Image/ExifTool/QuickTimeStream.pl in the Linux version to fix this in your copy before the new version comes out.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 11:29:02 AM
Hi Phil,

I tried using that file, no error and no data.

Code: [Select]
./exiftool -api largefilesupport -ee -p "$gpstimestamp,$gpslatitude,$gpslongitude"  "rawvideos/VID_002.insv" > VID_002.csv
Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 11:31:58 AM
1. Use single quotes on Linux.

2. Use GPSDateTime, not GPSTimeStamp

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 11:40:24 AM
How to get all param names?
I tried "$latitude,$longitude"
"$gpslatitude,$gpslongitude"

all null. I think the command works but no param with the name "$gpslatitude,$gpslongitude". And the same command works from windows.

Thanks,
yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 11:51:39 AM
This command should work for you:

./exiftool -api largefilesupport -ee -p '$gpsdatetime,$gpslatitude,$gpslongitude'  'rawvideos/VID_002.insv' > VID_002.csv

I tested it with a smaller INSV file and it works fine.

If you want to list all available tags, you can use this command:

./exiftool -s -ee rawvideos/VID_002.insv

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 12:05:59 PM
Hi Phil,

Thanks a lot for all your support.
I am getting results now.
However the result from Linux is different than the windows.
Linux: 2019:10:02 14:09:38Z,40 deg 38' 24.62" N,73 deg 58' 1.64" W
Windows: 2019:10:02 13:06:07Z 40.69177316 -73.99179094

Is it possible to get the same windows format from linux?

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 12:16:06 PM
You are using a config file on windows to change the default coordinate format.
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 03:02:14 PM
Hi Phil,

No I am not using any config in windows. Just downloaded the exe and executed.

Thanks,
yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 05:36:43 PM
You will only get that result with the Windows command if you use a config file.  There must be a default config file named ".ExifTool_config" in either your home directory or the exiftool app directory.

- Phil