Oh, it happens with servant on the client side. I use a swagger-codegen generated servant based client to access kubernetes. Calling https://github.com/soundcloud/haskell-kubernetes/blob/master/lib/Kubernetes/... works Calling https://github.com/soundcloud/haskell-kubernetes/blob/master/lib/Kubernetes/... doesn't.

From what I see the `QueryParam "labelSelector" Text` is the same in both.

Logging the value gives the literal `=` as expected. Logging the (http-client) requests shows the difference. Best Jan Nigel Rantor schrieb am Sa., 14. Jan. 2017 um 18:11 Uhr:

On 13/01/17 19:28, Jan von Löwenstein wrote:

...

Hi,

I have got a two places with a `QueryParam "q" Text` and call it with a Text that contains a `=` literal. In one place the `=` is correctly encoded as %3D, in the other I see a `!D(MISSING)`.

This has to happen somewhere in Servant or the lower layers. Both Texts print out nicely with an `=` sign if I just print them to stdout.

Google does not find `!D(MISSING)` anywhere.

Any idea what could possibly be the problem here?

It would be really useful to see exactly how you are 'calling' this.

Do you have a curl command line or are you using some other client?

n

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Grasping at straws a little bit here, but can you (a) do a packet capture (e.g. `tcpdump -X`) to see what's going back and forth on the wire, just to make absolutely sure all of the oddness is on the client's end? A string like `!D(MISSING)` sorta looks like a decoding error so perhaps it will be instructive to look at the bytes on the wire it's trying to decode, although I can't think what could be confused with an equals sign. It's not like that blooming Greek question mark ( http://www.fileformat.info/info/unicode/char/037e/index.htm). Cheers, On 14 January 2017 at 18:47, Nigel Rantor wrote:

On 14/01/17 17:37, Jan von Löwenstein wrote:

...

Oh, it happens with servant on the client side.

I use a swagger-codegen generated servant based client to access kubernetes. Calling https://github.com/soundcloud/haskell-kubernetes/blob/master /lib/Kubernetes/Api/ApivApi.hs#L475 works Calling https://github.com/soundcloud/haskell-kubernetes/blob/master /lib/Kubernetes/Api/ApivApi.hs#L473 doesn't.

From what I see the `QueryParam"labelSelector" Text`is the same in both.

Logging the value gives the literal `=` as expected. Logging the (http-client) requests shows the difference.

So I went ahead and wrote a simple client by hand and when I call the two functions on a local endpoint that shows me the request URL I get the following.

Same arguments for 'pretty' and 'labelSelector' in both cases.

Code:

runListSecret :: Manager -> BaseUrl -> ClientM SecretList

runListSecret manager baseUrl = listSecret (Just "prettyArg") (Just "=") Nothing Nothing Nothing Nothing manager baseUrl

runListService :: Manager -> BaseUrl -> ClientM ServiceList

runListService manager baseUrl = listService (Just "prettyArg") (Just "=") Nothing Nothing Nothing Nothing manager baseUrl

Server log:

127.0.0.1 - - [14/Jan/2017:18:39:49 +0000] "GET /api/v1/secrets?pretty=prettyArg&labelSelector=%3D HTTP/1.1" 404 - 0.0003 127.0.0.1 - - [14/Jan/2017:18:39:49 +0000] "GET /api/v1/services?pretty=prettyArg&labelSelector=%3D HTTP/1.1" 404 - 0.0003

I haven't used swagger-codegen but the servant-client code works for me so I would look more into that.

Swagger codegen looks like it might take me a while to get my head into so maybe you could look at the generated code, or sling it to me here or directly.

The two calls are so similar that I can't see why the generated client would behave differently in those cases.

Regards,

n

Best

...

Jan

Nigel Rantor mailto:wiggly@wiggly.org> schrieb am

Sa., 14. Jan. 2017 um 18:11 Uhr:

On 13/01/17 19:28, Jan von Löwenstein wrote: > Hi, > > I have got a two places with a `QueryParam "q" Text` and call it with a > Text that contains a `=` literal. In one place the `=` is correctly > encoded as %3D, in the other I see a `!D(MISSING)`. > > This has to happen somewhere in Servant or the lower layers. Both Texts > print out nicely with an `=` sign if I just print them to stdout. > > Google does not find `!D(MISSING)` anywhere. > > Any idea what could possibly be the problem here?

It would be really useful to see exactly how you are 'calling' this.

Do you have a curl command line or are you using some other client?

n

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

So, yes, I think it is in your client code Could you call both of them with *exactly* the same input? i.e. create a single selector and call both of them with exactly that input? I just feel like this might be a really silly typo or something somewhere. Alternatively can you give us the code you're using so we can play with it? n On 14/01/17 19:48, Jan von Löwenstein wrote:

Actually the code I pointed at _is_ the swagger-codegen generated client.

I have got a very small wrapper around it:

``` listService :: (MonadIO m, MonadCatch m, MonadReader Config m, MonadLog Text m) => Text -> m ServiceList.ServiceList listService labelSelector = namespacedF $ \namespace -> Kube.listNamespacedService namespace Nothing (Just labelSelector) Nothing Nothing Nothing Nothing

listSecret :: (MonadIO m, MonadThrow m, MonadReader Config m, MonadLog Text m) => Text -> m SecretList.SecretList listSecret labelSelector = namespacedF $ \namespace -> Kube.listNamespacedSecret namespace Nothing (Just labelSelector) Nothing Nothing Nothing Nothing

namespacedF :: (MonadIO m, MonadThrow m, MonadReader Config m, MonadLog Text m) => NamespacedF model -> m model namespacedF f = do config <- ask result <- let baseUrl = apiEndpoint config tlsSettings = Base.tlsSettings baseUrl (credentials config) kubeNamespace = namespace config in do manager <- liftIO $ newManager $ (mkManagerSettings tlsSettings Nothing) { managerModifyRequest = \req -> return req { checkResponse = \req res -> do Text.IO.hPutStrLn stderr (Text.pack (show req)) responseMessage <- showResponse res Text.IO.hPutStrLn stderr responseMessage return () } } liftIO $ runExceptT $ f kubeNamespace manager baseUrl either throwM return result ```

I call them like: ``` do let Just agentId = ... selector = "agentId" <> "=" <> agentId logMessage $ "listSecret selector: '" <> selector <> "'" secretList <- listSecret selector ```

`logMessage` gives a literal equals sign in both cases. `Text.IO.hPutStrLn stderr (Text.pack (show req))` from namespacedF gives `%3D` and `%!D(MISSING)` depending on which function I call.

From what I understand this is definitely happening on the client side. Do you still think a tcpdump might be worth it?

``` listSecret selector: 'agentId=b49d4406-8020-44e6-7788-6f92d5c0e732' Request { host = "192.168.99.100" port = 8443 secure = True requestHeaders = [("Accept","application/json")] path = "/api/v1/namespaces/default/secrets" queryString = "?labelSelector=agentId%!D(MISSING)b49d4406-8020-44e6-7788-6f92d5c0e732" method = "GET" proxy = Nothing rawBody = False redirectCount = 10 responseTimeout = ResponseTimeoutDefault requestVersion = HTTP/1.1 } ```

``` listService selector: 'agentId=24f99a4b-1682-44da-7ba4-dba935d107d2' Request { host = "192.168.99.100" port = 8443 secure = True requestHeaders = [("Accept","application/json")] path = "/api/v1/namespaces/default/services" queryString = "?labelSelector=agentId%3D24f99a4b-1682-44da-7ba4-dba935d107d2" method = "GET" proxy = Nothing rawBody = False redirectCount = 10 responseTimeout = ResponseTimeoutDefault requestVersion = HTTP/1.1 } ```

Thanks for the time you already invested. I am completely puzzled because I see no chance how the code could behave as it does. (From my experience that points out I am looking at the wrong place ;) )

Best Jan

David Turner mailto:dct25-561bs@mythic-beasts.com> schrieb am Sa., 14. Jan. 2017 um 20:08 Uhr:

Grasping at straws a little bit here, but can you (a) do a packet capture (e.g. `tcpdump -X`) to see what's going back and forth on the wire, just to make absolutely sure all of the oddness is on the client's end?

A string like `!D(MISSING)` sorta looks like a decoding error so perhaps it will be instructive to look at the bytes on the wire it's trying to decode, although I can't think what could be confused with an equals sign. It's not like that blooming Greek question mark (http://www.fileformat.info/info/unicode/char/037e/index.htm).

Cheers,

On 14 January 2017 at 18:47, Nigel Rantor mailto:wiggly@wiggly.org> wrote:

On 14/01/17 17:37, Jan von Löwenstein wrote:

Oh, it happens with servant on the client side.

I use a swagger-codegen generated servant based client to access kubernetes. Calling https://github.com/soundcloud/haskell-kubernetes/blob/master/lib/Kubernetes/... works Calling https://github.com/soundcloud/haskell-kubernetes/blob/master/lib/Kubernetes/... doesn't.

From what I see the `QueryParam"labelSelector" Text`is the same in both.

Logging the value gives the literal `=` as expected. Logging the (http-client) requests shows the difference.

So I went ahead and wrote a simple client by hand and when I call the two functions on a local endpoint that shows me the request URL I get the following.

Same arguments for 'pretty' and 'labelSelector' in both cases.

Code:

runListSecret :: Manager -> BaseUrl -> ClientM SecretList

runListSecret manager baseUrl = listSecret (Just "prettyArg") (Just "=") Nothing Nothing Nothing Nothing manager baseUrl

runListService :: Manager -> BaseUrl -> ClientM ServiceList

runListService manager baseUrl = listService (Just "prettyArg") (Just "=") Nothing Nothing Nothing Nothing manager baseUrl

Server log:

127.0.0.1 - - [14/Jan/2017:18:39:49 +0000] "GET /api/v1/secrets?pretty=prettyArg&labelSelector=%3D HTTP/1.1" 404 - 0.0003 127.0.0.1 - - [14/Jan/2017:18:39:49 +0000] "GET /api/v1/services?pretty=prettyArg&labelSelector=%3D HTTP/1.1" 404 - 0.0003

I haven't used swagger-codegen but the servant-client code works for me so I would look more into that.

Swagger codegen looks like it might take me a while to get my head into so maybe you could look at the generated code, or sling it to me here or directly.

The two calls are so similar that I can't see why the generated client would behave differently in those cases.

Regards,

n

Best Jan

Nigel Rantor mailto:wiggly@wiggly.org mailto:wiggly@wiggly.org>> schrieb am

Sa., 14. Jan. 2017 um 18:11 Uhr:

On 13/01/17 19:28, Jan von Löwenstein wrote: > Hi, > > I have got a two places with a `QueryParam "q" Text` and call it with a > Text that contains a `=` literal. In one place the `=` is correctly > encoded as %3D, in the other I see a `!D(MISSING)`. > > This has to happen somewhere in Servant or the lower layers. Both Texts > print out nicely with an `=` sign if I just print them to stdout. > > Google does not find `!D(MISSING)` anywhere. > > Any idea what could possibly be the problem here?

It would be really useful to see exactly how you are 'calling' this.

Do you have a curl command line or are you using some other client?

n

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to:

http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

On 14/01/17 21:03, Brandon Allbery wrote:

On Sat, Jan 14, 2017 at 8:34 PM, Nigel Rantor mailto:wiggly@wiggly.org> wrote:

So, yes, I think it is in your client code

On very little evidence (mostly the ! prefix) I am wondering if the client is using an XML library and it's glitching for some reason.

These are query parameters within the URL that is being requested, I'm not sure why any particular content-type being used would have an effect here. Regards, n