streaming-commons-0.1.17: Common lower-level functions needed by various streaming data libraries

Safe Haskell | None
Language | Haskell98

Data.Streaming.ByteString.Builder

Contents

- toByteStringIO
- Buffers
  - Status information
  - Creation and modification
  - Conversion to bytestings
- Buffer allocation strategies

Description

Convert a stream of bytestring `Builder`s into a stream of `ByteString`s.

Adapted from blaze-builder-enumerator, written by Michael Snoyman and Simon Meier.

Note that the functions here can work in any monad built on top of `IO` or `ST`.

Also provides `toByteStringIO*` like Blaze.ByteString.Builders, for Data.ByteString.Builder.

Since 0.1.9

Synopsis

- type BuilderRecv = Builder -> IO BuilderPopper
- type BuilderPopper = IO ByteString
- type BuilderFinish = IO (Maybe ByteString)
- newByteStringBuilderRecv :: BufferAllocStrategy -> IO (BuilderRecv, BuilderFinish)
- toByteStringIO :: (ByteString -> IO ()) -> Builder -> IO ()
- toByteStringIOWith :: Int -> (ByteString -> IO ()) -> Builder -> IO ()
- toByteStringIOWithBuffer :: Int -> (ByteString -> IO ()) -> Builder -> ForeignPtr Word8 -> IO ()
- data Buffer
- freeSize :: Buffer -> Int
- sliceSize :: Buffer -> Int
- bufferSize :: Buffer -> Int
- allocBuffer :: Int -> IO Buffer
- reuseBuffer :: Buffer -> Buffer
- nextSlice :: Int -> Buffer -> Maybe Buffer
- unsafeFreezeBuffer :: Buffer -> ByteString
- unsafeFreezeNonEmptyBuffer :: Buffer -> Maybe ByteString
- type BufferAllocStrategy = (IO Buffer, Int -> Buffer -> IO (IO Buffer))
- allNewBuffersStrategy :: Int -> BufferAllocStrategy
- reuseBufferStrategy :: IO Buffer -> BufferAllocStrategy
- defaultStrategy :: BufferAllocStrategy

# Documentation

type BuilderRecv = Builder -> IO BuilderPopper #

type BuilderPopper = IO ByteString #

Provides a series of `ByteString`s until empty, at which point it provides an empty `ByteString`.

Since 0.1.10.0

type BuilderFinish = IO (Maybe ByteString) #

newByteStringBuilderRecv :: BufferAllocStrategy -> IO (BuilderRecv, BuilderFinish) #

# toByteStringIO

toByteStringIO :: (ByteString -> IO ()) -> Builder -> IO () #

Run the builder with a `defaultChunkSize`d buffer and execute the given `IO` action whenever the buffer is full or gets flushed.

     toByteStringIO = toByteStringIOWith defaultChunkSize

Since 0.1.9

toByteStringIOWith #

Arguments

:: Int | Buffer size (upper bounds the number of bytes forced per call to the `IO` action).
-> (ByteString -> IO ()) | `IO` action to execute per full buffer, which is referenced by a strict `ByteString`.
-> Builder | `Builder` to run.
-> IO () |

`toByteStringIOWith bufSize io b` runs the builder `b` with a buffer of at least the size `bufSize` and executes the `IO` action `io` whenever the buffer is full.

Compared to `toLazyByteStringWith` this function requires less allocation, as the output buffer is only allocated once at the start of the serialization and whenever something bigger than the current buffer size has to be copied into the buffer, which should happen very seldomly for the default buffer size of 32kb. Hence, the pressure on the garbage collector is reduced, which can be an advantage when building long sequences of bytes.

Since 0.1.9

toByteStringIOWithBuffer :: Int -> (ByteString -> IO ()) -> Builder -> ForeignPtr Word8 -> IO () #

Use a pre-existing buffer to `toByteStringIOWith`.

Since 0.1.9

# Buffers

data Buffer #

A buffer `Buffer fpbuf p0 op ope` describes a buffer with the underlying byte array `fpbuf..ope`, the currently written slice `p0..op` and the free space `op..ope`.

Since 0.1.10.0

## Status information

freeSize :: Buffer -> Int #

The size of the free space of the buffer.

Since 0.1.10.0

sliceSize :: Buffer -> Int #

The size of the written slice in the buffer.

Since 0.1.10.0

bufferSize :: Buffer -> Int #

The size of the whole byte array underlying the buffer.

Since 0.1.10.0

## Creation and modification

allocBuffer :: Int -> IO Buffer #

`allocBuffer size` allocates a new buffer of size `size`.

Since 0.1.10.0

reuseBuffer :: Buffer -> Buffer #

Resets the beginning of the next slice and the next free byte such that the whole buffer can be filled again.

Since 0.1.10.0

nextSlice :: Int -> Buffer -> Maybe Buffer #

Move the beginning of the slice to the next free byte such that the remaining free space of the buffer can be filled further. This operation is safe and can be used to fill the remaining part of the buffer after a direct insertion of a bytestring or a flush.

Since 0.1.10.0

## Conversion to bytestings

unsafeFreezeBuffer :: Buffer -> ByteString #

Convert the buffer to a bytestring. This operation is unsafe in the sense that created bytestring shares the underlying byte array with the buffer. Hence, depending on the later use of this buffer (e.g., if it gets reset and filled again) referential transparency may be lost.

Since 0.1.10.0

unsafeFreezeNonEmptyBuffer :: Buffer -> Maybe ByteString #

Convert a buffer to a non-empty bytestring. See `unsafeFreezeBuffer` for the explanation of why this operation may be unsafe.

Since 0.1.10.0

# Buffer allocation strategies

type BufferAllocStrategy = (IO Buffer, Int -> Buffer -> IO (IO Buffer)) #

A buffer allocation strategy `(buf0, nextBuf)` specifies the initial buffer to use and how to compute a new buffer `nextBuf minSize buf` with at least size `minSize` from a filled buffer `buf`. The double nesting of the `IO` monad helps to ensure that the reference to the filled buffer `buf` is lost as soon as possible, but the new buffer doesn't have to be allocated too early.

Since 0.1.10.0

allNewBuffersStrategy :: Int -> BufferAllocStrategy #

The simplest buffer allocation strategy: whenever a buffer is requested, allocate a new one that is big enough for the next build step to execute.

NOTE that this allocation strategy may spill quite some memory upon direct insertion of a bytestring by the builder. Thats no problem for garbage collection, but it may lead to unreasonably high memory consumption in special circumstances.

Since 0.1.10.0

reuseBufferStrategy :: IO Buffer -> BufferAllocStrategy #

An unsafe, but possibly more efficient buffer allocation strategy: reuse the buffer, if it is big enough for the next build step to execute.

Since 0.1.10.0

defaultStrategy :: BufferAllocStrategy #