TotalDepth.BIT.ReadBIT¶

This processes Dresser Atlas BIT files. Dresser Atlas BIT files are TIF encoded and consist of a set of Log Passes:

First block, this gives a description of the file.
Subsequent blocks are frame data.

TIF markers are used to separate Log Passes and delineate the file. A TIF marker type 1 ends the Log Pass and a pair of type 1 markers ends the readable file.

Here is an example of TIF markers in a file:

$ tddetif -n data/DresserAtlasBIT/special/29_10-_3Z/DWL_FILE/29_10-_3Z_dwl_DWL_WIRE_1644659.bit -v
Cmd: DWL_FILE/29_10-_3Z_dwl_DWL_WIRE_1644659.bit -v
Detected 187 TIF Markers in data/DresserAtlasBIT/special/29_10-_3Z/DWL_FILE/29_10-_3Z_dwl_DWL_WIRE_1644659.bit
[     0] TifMarker: 0x00000000 Type: 0x00000000 Prev: 0x00000000 Next: 0x00000120 Length: 0x00000120 Payload: 0x00000114
[     1] TifMarker: 0x00000120 Type: 0x00000000 Prev: 0x00000000 Next: 0x000003ac Length: 0x0000028c Payload: 0x00000280
[     2] TifMarker: 0x000003ac Type: 0x00000000 Prev: 0x00000120 Next: 0x00000638 Length: 0x0000028c Payload: 0x00000280
...
[    91] TifMarker: 0x0000e658 Type: 0x00000000 Prev: 0x0000e3cc Next: 0x0000e8e4 Length: 0x0000028c Payload: 0x00000280
[    92] TifMarker: 0x0000e8e4 Type: 0x00000000 Prev: 0x0000e658 Next: 0x0000eb70 Length: 0x0000028c Payload: 0x00000280
[    93] TifMarker: 0x0000eb70 Type: 0x00000001 Prev: 0x0000e8e4 Next: 0x0000eb7c Length: 0x0000000c Payload: 0x00000000
[    94] TifMarker: 0x0000eb7c Type: 0x00000000 Prev: 0x0000eb70 Next: 0x0000ec9c Length: 0x00000120 Payload: 0x00000114
[    95] TifMarker: 0x0000ec9c Type: 0x00000000 Prev: 0x0000eb7c Next: 0x0000ef28 Length: 0x0000028c Payload: 0x00000280
...
[   184] TifMarker: 0x0001cf48 Type: 0x00000000 Prev: 0x0001ccbc Next: 0x0001d1d4 Length: 0x0000028c Payload: 0x00000280
[   185] TifMarker: 0x0001d1d4 Type: 0x00000001 Prev: 0x0001cf48 Next: 0x0001d1e0 Length: 0x0000000c Payload: 0x00000000
[   186] TifMarker: 0x0001d1e0 Type: 0x00000001 Prev: 0x0001d1d4 Next: 0x0001d1ec Length: 0x0000000c Payload: 0x00000000
Execution time =    0.020 (S)
Bye, bye!

Example first block without TIF markers, length 276 (0x114) bytes:

0000000c: 0002 0000 5348 454c 4c20 4558 5052 4f20  ....SHELL EXPRO
0000001c: 552e 4b2e 2020 2020 2020 3234 204f 4354  U.K.      24 OCT
0000002c: 2038 3420 2020 2020 204d 414e 5346 4945   84      MANSFIE
0000003c: 4c44 2f44 4f44 4453 2020 2020 2020 2020  LD/DODDS
0000004c: 2020 2020 2020 2020 2020 2020 000a 0018              ....
0000005c: 0054 2020 3220 3920 2f20 3120 3020 2d20  .T  2 9 / 1 0 -
0000006c: 3320 2020 2020 2020 2020 2020 2020 2020  3
0000007c: 2020 2020 2020 2020 2020 2020 2020 2020
0000008c: 2020 2020 2020 2020 2020 2020 2020 2020
0000009c: 2020 2020 2020 2020 2020 2020 0012 000b              ....
000000ac: 0006 2020 000a 0000 434f 4e44 534e 2020  ..  ....CONDSN
000000bc: 5350 2020 4752 2020 4341 4c20 5445 4e20  SP  GR  CAL TEN
000000cc: 5350 4420 4143 5120 4143 2020 5254 2020  SPD ACQ AC  RT
000000dc: 2020 2020 2020 2020 2020 2020 2020 2020
000000ec: 2020 2020 2020 2020 2020 2020 2020 2020
000000fc: 2020 2020 2020 2020 443a 6600 4438 fe00          D:f.D8..
0000010c: 4040 0000 0000 0000 4210 0000 4d4e 3233  @@......B...MN23
0000011c: 394a 2031

Decomposed:

4 bytes unknown.
160 bytes ASCII description, there is some structure here but it is as yet unknown.
C, the count of channels as big endian two byte format ‘>H’ or ‘>h’.
A two byte null
Channel names, 4 bytes each, this is always 80 bytes long.
Five 4 byte floats start, stop, step, 0, ???.
Unknown tail of eight bytes.

Total = 4 + 160 + 2 + 2 + 80 + 5 * 4 + 8 = 276 (0x114) bytes.

Of note is that while the TIF markers are little endian many values within the file are big endian.

Decoding the Description¶

This is 160 bytes long. Example:

$ xxd -s +16 data/DresserAtlasBIT/special/29_10-_3Z/DWL_FILE/29_10-_3Z_dwl_DWL_WIRE_1644659.bit | head -n 20
00000010: 5348 454c 4c20 4558 5052 4f20 552e 4b2e  SHELL EXPRO U.K.
00000020: 2020 2020 2020 3234 204f 4354 2038 3420        24 OCT 84
00000030: 2020 2020 204d 414e 5346 4945 4c44 2f44       MANSFIELD/D
00000040: 4f44 4453 2020 2020 2020 2020 2020 2020  ODDS
00000050: 2020 2020 2020 2020 000a 0018 0054 2020          .....T
00000060: 3220 3920 2f20 3120 3020 2d20 3320 2020  2 9 / 1 0 - 3
00000070: 2020 2020 2020 2020 2020 2020 2020 2020
00000080: 2020 2020 2020 2020 2020 2020 2020 2020
00000090: 2020 2020 2020 2020 2020 2020 2020 2020
000000a0: 2020 2020 2020 2020 0012 000b 0006 2020          ......
000000b0: 000a 0000 434f 4e44 534e 2020 5350 2020  ....CONDSN  SP

Looks like we have 4 * 16 + 8 = 72 bytes of ASCII to 0x58 as a description.

Either:

Then 24 bytes of binary data to 0x70 ???:

000a 0018 0054 2020 3220 3920 2f20 3120 3020 2d20 3320 2020

3 * 16 + 8 = 56 ASCII spaces. hmm divided by 4 is 19. hmm maybe together 24 + 56 = 80 and 80 / 4 is 20. Channel units?

Then 8 bytes of stuff: 0012 000b 0006 2020

Total is: 72 + 24 + 56 + 8 == 160

Or, alternatively: 72 (4 * 16 + 8) bytes of ASCII. Then five bytes: 000a 0018 00 Then 75 bytes of ASCII: 54 2020 3220 3920 2f20 3120 3020 2d20 3320 2020 … Then 8 bytes of stuff: 0012 000b 0006 2020 Total is: 72 + 5 + 75 + 8 == 160

Decoding the frames¶

Each subsequent block is a subdivided into the number of channels and read into that channel. For example a block of data 0x280 (640) bytes long with 10 channels is decomposed into 10 sub-blocks of 64 bytes each. Each sub-block contains 16 floats.

There is a directory that has both BIT and LIS files in it.

LIS: 29_10-_3Z/DWL_FILE/29_10-_3Z_dwl_DWL_WIRE_1988494.lis BIT: 29_10-_3Z_dwl_DWL_WIRE_1644659.bit and 29_10-_3Z_dwl_DWL_WIRE_1644660.bit

Log Pass 0 has X axis: 14950.000 (FT) to 14582.250 (FT) Interval -367.750 (FT) Total number of frames 1472 Overall frame spacing -0.250 (FT)

Corresponding BIT file 29_10-_3Z_dwl_DWL_WIRE_1644659.bit has:

LogPassRange(depth_from=14950.000891089492, depth_to=14590.000869631818, spacing=0.2500000149011621, unknown_a=0.0,
unknown_b=16.000000953674373)

Frames from spacing: 1441

A striking feature of the LIS Log Pass 0 file is that the SP is fixed throughout at -249.709. This value appears nowhere else in the LIS Log Pass. The BIT file 29_10-_3Z_dwl_DWL_WIRE_1644659.bit corresponds with the following:

The binary data is assumed to start at 0x128 + 4
Each channel is sequential but read in blocks of 16 floats (64 bytes).
After 16 floats are read for each channel (every 160 floats, or 640 bytes) then 12 bytes are read and discarded.
Although the BIT file states 1441 frames from spacing the LIS file has read 1472 (0x5c0) frames (modulo 16) with the

remaining values as 0.0001 for all channels.

The 12 bytes read after every 640 bytes look like this: 4 nulls, two values unknown, two nulls, two values unknown, two nulls. These are TIF markers.

The 0.0001 figure is actually 9.999999615829415e-05 or 0x3d 0x68 0xdb 0x8b

exception TotalDepth.BIT.ReadBIT.ExceptionTotalDepthBIT¶: Simple specialisation of an exception class for TotalDepth.BIT.

exception TotalDepth.BIT.ReadBIT.ExceptionTotalDepthBIT_TIF¶: When TIF markers go wrong.

exception TotalDepth.BIT.ReadBIT.ExceptionTotalDepthBITFirstBlock¶: Constructor from first block of data.

exception TotalDepth.BIT.ReadBIT.ExceptionTotalDepthBITDataBlocks¶: Constructor from first block of data.

TotalDepth.BIT.ReadBIT.bytes_to_float(b: bytes) → float¶

Returns a float from four bytes.

https://en.wikipedia.org/wiki/IBM_hexadecimal_floating-point

Example: -118.625 -> b’Âv ‘.

NOTE: This is the same as RP66V1 ISINGL (5) Representation Code.

TotalDepth.BIT.ReadBIT.float_to_bytes(f: float) → bytes¶

Returns four bytes from a float.

https://en.wikipedia.org/wiki/IBM_hexadecimal_floating-point

Example: b’Âv ‘ -> -118.625

NOTE: This is the same as RP66V1 ISINGL (5) Representation Code.

TotalDepth.BIT.ReadBIT.read_float(file: BinaryIO) → float¶: Returns a float from the current read position.

TotalDepth.BIT.ReadBIT.gen_floats(b: bytes) → Sequence[float]¶: Yields a sequence of floats from the bytes.

TotalDepth.BIT.ReadBIT.is_bit_file(fobj: BinaryIO) → bool¶: Returns True the file looks like a Western Atlas BIT file, False otherwise.

TotalDepth.BIT.ReadBIT.is_bit_file_from_path(file_path: str) → bool¶: Returns True the file looks like a Western Atlas BIT file, False otherwise.

class TotalDepth.BIT.ReadBIT.TifMarker¶

Contains a TIF marker with its file position.

tell¶: Alias for field number 0

type¶: Alias for field number 1

prev¶: Alias for field number 2

next¶: Alias for field number 3

__getnewargs__()¶: Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, tell: int, type: int, prev: int, next: int)¶: Create new instance of TifMarker(tell, type, prev, next)

__repr__()¶: Return a nicely formatted representation string

class TotalDepth.BIT.ReadBIT.TifType¶: Type of TIF marker. Type 0 is normal data. Type 1 is end of Log Pass. Type 2 is end of file.

class TotalDepth.BIT.ReadBIT.TifMarkedBytes¶

This is yielded by yield_tif_blocks().

tell¶: Alias for field number 0

tif_type¶: Alias for field number 1

payload¶: Alias for field number 2

__getnewargs__()¶: Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, tell: int, tif_type: TotalDepth.BIT.ReadBIT.TifType, payload: bytes)¶: Create new instance of TifMarkedBytes(tell, tif_type, payload)

__repr__()¶: Return a nicely formatted representation string

TotalDepth.BIT.ReadBIT.yield_tif_blocks(file: BinaryIO) → Sequence[TotalDepth.BIT.ReadBIT.TifMarkedBytes]¶: Generate the payload from blocks of the file.

class TotalDepth.BIT.ReadBIT.LogPassRange¶

POD container for the five floats that describe the range of the BIT Log Pass.

depth_from¶: Alias for field number 0

depth_to¶: Alias for field number 1

spacing¶: Alias for field number 2

unknown_a¶: Alias for field number 3

unknown_b¶: Alias for field number 4

__getnewargs__()¶: Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, depth_from: float, depth_to: float, spacing: float, unknown_a: float, unknown_b: float)¶: Create new instance of LogPassRange(depth_from, depth_to, spacing, unknown_a, unknown_b)

__repr__()¶: Return a nicely formatted representation string

TotalDepth.BIT.ReadBIT.read_bytes_from_offset(b: bytes, count: int, offset: int) → Tuple[bytes, int]¶

Slices a bytes object and increments the offset. Usage:

result, offset = read_bytes_from_offset(b, count, offset)

class TotalDepth.BIT.ReadBIT.BITFrameArray(ident: str, tif_block: TotalDepth.BIT.ReadBIT.TifMarkedBytes)¶

Represents a Log Pass from a BIT file. This has a number of fields, some are BIT specific but self.frame_array is a TotalDepth.common.LogPass.FrameArray.

__init__(ident: str, tif_block: TotalDepth.BIT.ReadBIT.TifMarkedBytes)¶

Example initial block, length 0x114, 276 bytes:

0000000c: 0002 0000 5348 454c 4c20 4558 5052 4f20  ....SHELL EXPRO
0000001c: 552e 4b2e 2020 2020 2020 3234 204f 4354  U.K.      24 OCT
0000002c: 2038 3420 2020 2020 204d 414e 5346 4945   84      MANSFIE
0000003c: 4c44 2f44 4f44 4453 2020 2020 2020 2020  LD/DODDS
0000004c: 2020 2020 2020 2020 2020 2020 000a 0018              ....
0000005c: 0054 2020 3220 3920 2f20 3120 3020 2d20  .T  2 9 / 1 0 -
0000006c: 3320 2020 2020 2020 2020 2020 2020 2020  3
0000007c: 2020 2020 2020 2020 2020 2020 2020 2020
0000008c: 2020 2020 2020 2020 2020 2020 2020 2020
0000009c: 2020 2020 2020 2020 2020 2020 0012 000b              ....
000000ac: 0006 2020 000a 0000 434f 4e44 534e 2020  ..  ....CONDSN
000000bc: 5350 2020 4752 2020 4341 4c20 5445 4e20  SP  GR  CAL TEN
000000cc: 5350 4420 4143 5120 4143 2020 5254 2020  SPD ACQ AC  RT
000000dc: 2020 2020 2020 2020 2020 2020 2020 2020
000000ec: 2020 2020 2020 2020 2020 2020 2020 2020
000000fc: 2020 2020 2020 2020 443a 6600 4438 fe00          D:f.D8..
0000010c: 4040 0000 0000 0000 4210 0000 4d4e 3233  @@......B...MN23
0000011c: 394a 2031

long_str() → str¶: Returns a multi-line string describing self.

add_block(block: bytes) → None¶: Adds a data block of frame data to my temporary data structure(s).

complete() → None¶: Converts the existing frame data to a LogPass.FrameArray. This adds a computed X-axis and removes temporary data structures.

__weakref__¶: list of weak references to the object (if defined)

TotalDepth.BIT.ReadBIT.create_bit_frame_array_from_file(file: BinaryIO) → List[TotalDepth.BIT.ReadBIT.BITFrameArray]¶: Given a file this returns a list of BITFrameArray objects.

TotalDepth.BIT.ReadBIT.create_bit_frame_array_from_path(file_path: str) → List[TotalDepth.BIT.ReadBIT.BITFrameArray]¶: Given a file path this returns a list of BITFrameArray objects or raises.

class TotalDepth.BIT.ReadBIT.FileSizeTime(name, size, time)¶

name¶: Alias for field number 0

size¶: Alias for field number 1

time¶: Alias for field number 2

__str__()¶: Return str(self).

__getnewargs__()¶: Return self as a plain tuple. Used by copy and pickle.

static __new__(_cls, name: str, size: int, time: float)¶: Create new instance of FileSizeTime(name, size, time)

__repr__()¶: Return a nicely formatted representation string

TotalDepth.BIT.ReadBIT.print_summarise_frame_array(frame_array: TotalDepth.BIT.ReadBIT.BITFrameArray) → None¶: Summarise all the channels in the Frame Array.

TotalDepth.BIT.ReadBIT.print_process_file(file_path: str, verbose: int, summary: bool) → TotalDepth.BIT.ReadBIT.FileSizeTime¶: Process a BIT file and print out a summary.

TotalDepth.BIT.ReadBIT.print_process_directory(directory: str, recursive: bool, verbose: int, summary: bool) → Dict[str, TotalDepth.BIT.ReadBIT.FileSizeTime]¶: Processes a complete directory for BIT files.

TotalDepth.BIT.ReadBIT.main() → int¶: Main entry point.