On Mon, Nov 28, 2022 at 08:42:40PM +0100, Hécate wrote:
Thanks for the knowledge regarding the size of the ByteString!
Eventually the outside serialisation isn't up to me, but rather the consumers. I have one potential user who told me they need the signature and the message to be separate for the protocol they are implementing.
I guess if I defer to ByteString early on it'll be easier for the library to be adapted to whatever use case comes after.
Note that "Lazy" Bytestrings are allocated as a chain of component ByteString chunks, and their total length is an Int64 on all platforms. The chunks might still be in memory, and in some cases could be truly incrementally loaded from the underlying source. However true streaming support is perhaps best via ByteStream from e.g. https://hackage.haskell.org/package/streaming-bytestring-0.2.4/docs/Streamin... which supports monadic incremental consumption (the underlying Monad need not be IO, so pure sources are also supported). The design space is wide. If signatures are "detached" (available independently of the message), and messages could be too large to fit in memory, then streaming the message may be attractive. But then the protocol to the consumer needs to support streaming input (e.g. HTTP PUT or POST with chunked transfer encoding). With large messages one might even want chunk-level signatures that authenticate all the chunks transmitted so far in order, with a final empty chunk authenticating the entire message. There should be a standard format for this, but I am not aware of one. CMS sadly does not suffice. For example, messages that are just signed, but not cryptographically tied to a particular transaction (and recipient) are potentially subject to replays. This may or may not be OK. A message with a clear transaction context (that makes out of context replays detectable), encrypted to a given set of recipients, and then signed has stronger security properties than just a signed blob that is not tied to a clear context or recipient. It is sadly common to use cryptography pixie dust to add security theatre to application data flows without a careful analysis of the security properties required and attained. Security is difficult and brittle. :-( -- Viktor.