Re: [Haskell-beginners] Nonsensical behavior with http-conduit, httpLbs

10 Apr 2014

      Yeap, it was elasticsearch.

Had to call
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indice...
force update on indexing of documents.

Sorry to bother everybody ;_;

Thank you so much for your help.

Have a great day :)

--- Chris

On Thu, Apr 10, 2014 at 1:28 AM, Michael Snoyman wrote:
...
http-conduit does not perform any kind of automatic concurrency. httpLbs
will only return after it has fully read the response body from the server.
So unless there's a bug somewhere in http-conduit, it seems that the server
has not fully updated its index by the time it gives back a response.
On Thu, Apr 10, 2014 at 9:25 AM, Christopher Allen wrote:
...
the function actually running the requests is:
dispatch :: String -> Method -> Maybe L.ByteString
            -> IO Reply
dispatch url method body = do
  initReq <- parseUrl url
  let reqBody = RequestBodyLBS $ fromMaybe emptyBody body
  let req = initReq { method = method
                    , requestBody = reqBody
                    , checkStatus = \_ _ _ -> Nothing}
  withManager $ httpLbs req
rest of the code is here: https://github.com/bitemyapp/bloodhound/
On Thu, Apr 10, 2014 at 1:25 AM, Christopher Allen wrote:
...
It's a step removed from that, and `seq`'ing insertData isn't fixing it,
only a threadDelay is.
The operations insertData is performing are:
delete index, create index, add document to index
the operation queryTweet is performing is:
search index
The behavior seems to indicate "search index" is happening before "add
document to index" and after "delete index", as the document will not exist
when the query is performed.
Only adding a threadDelay as reliably worked. Even `seq`'ing insertData
doesn't work.
Are the requests somehow being performed concurrently?
On Thu, Apr 10, 2014 at 1:15 AM, Michael Snoyman wrote:
...
I can't say I fully understand your code, but it seems like it's doing
the following:
1. Send request to server to delete data.
2. Send request to server to add data.
3. Request data from server.
You're saying that with a long enough delay between steps 2 and 3, then
(3) works, otherwise it fails. It sounds to me like there's some kind of a
race condition. I don't know anything about createExampleIndex, but are you
certain that it only returns after the data is completely ready for
querying?
On Thu, Apr 10, 2014 at 8:43 AM, Christopher Allen wrote:
...
to add to my question adding a delay made it work:
main :: IO ()
main = do
  _ <- insertData
  threadDelay 1000000
  myTweet <- queryTweet
  print myTweet
On Thu, Apr 10, 2014 at 12:41 AM, Christopher Allen 
...
wrote:
...
Repro'd on GHC 7.8.1 and 7.6.3
Original code is: https://github.com/bitemyapp/bloodhound/
The code only works if I manually (out of band) run insertData (in a
REPL) and then run main with the insertData invocation stripped out. If I
run main with the insertData invocation included, it throws an exception
(head) because the search results are empty.
The behavior is as if queryTweet was executing after insertData
deleted the index, but before it inserted the new data.
The following is a stripped down example:
insertData :: IO ()
insertData = do
  let encoded = encode exampleTweet
  _ <- deleteExampleIndex
  created <- createExampleIndex
  docCreated <- indexDocument (Server "http://localhost:9200")
"twitter" "tweet" exampleTweet "1"
  print "test"
  return ()
el
queryTweet :: IO (Either String Tweet)
queryTweet = do
  let queryFilter = BoolFilter (MustMatch (Term "user" "bitemyapp")
False)
                    <||> IdentityFilter
  let search = Search Nothing (Just queryFilter)
  reply <- searchByIndex testServer "twitter" search
  let result = eitherDecode (responseBody reply) :: Either String
(SearchResult Tweet)
  let myTweet = fmap (hitSource . head . hits . searchHits) result
  return myTweet
main :: IO ()
main = do
  _ <- insertData
  myTweet <- queryTweet
  print myTweet
further up the call chain, the http call is getting dispatched with
http-conduit's gear, the result returned with the expression:
withManager $ httpLbs req
Included this in case it affects the semantics.
Can anybody help? This has stumped me for a couple hours and I
couldn't get anything clueful on IRC.
Thanks to any that help.
--- Chris
_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners
_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners
_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners
_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners