Yeap, it was elasticsearch.

Had to call http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-refresh.html to force update on indexing of documents.

Sorry to bother everybody ;_;

Thank you so much for your help.

Have a great day :)

--- Chris



On Thu, Apr 10, 2014 at 1:28 AM, Michael Snoyman <michael@snoyman.com> wrote:
http-conduit does not perform any kind of automatic concurrency. httpLbs will only return after it has fully read the response body from the server. So unless there's a bug somewhere in http-conduit, it seems that the server has not fully updated its index by the time it gives back a response.


On Thu, Apr 10, 2014 at 9:25 AM, Christopher Allen <cma@bitemyapp.com> wrote:
the function actually running the requests is:

dispatch :: String -> Method -> Maybe L.ByteString
            -> IO Reply
dispatch url method body = do
  initReq <- parseUrl url
  let reqBody = RequestBodyLBS $ fromMaybe emptyBody body
  let req = initReq { method = method
                    , requestBody = reqBody
                    , checkStatus = \_ _ _ -> Nothing}
  withManager $ httpLbs req

rest of the code is here: https://github.com/bitemyapp/bloodhound/


On Thu, Apr 10, 2014 at 1:25 AM, Christopher Allen <cma@bitemyapp.com> wrote:
It's a step removed from that, and `seq`'ing insertData isn't fixing it, only a threadDelay is.

The operations insertData is performing are:
delete index, create index, add document to index

the operation queryTweet is performing is:
search index

The behavior seems to indicate "search index" is happening before "add document to index" and after "delete index", as the document will not exist when the query is performed.

Only adding a threadDelay as reliably worked. Even `seq`'ing insertData doesn't work.

Are the requests somehow being performed concurrently?



On Thu, Apr 10, 2014 at 1:15 AM, Michael Snoyman <michael@snoyman.com> wrote:
I can't say I fully understand your code, but it seems like it's doing the following:

1. Send request to server to delete data.
2. Send request to server to add data.
3. Request data from server.

You're saying that with a long enough delay between steps 2 and 3, then (3) works, otherwise it fails. It sounds to me like there's some kind of a race condition. I don't know anything about createExampleIndex, but are you certain that it only returns after the data is completely ready for querying?


On Thu, Apr 10, 2014 at 8:43 AM, Christopher Allen <cma@bitemyapp.com> wrote:
to add to my question adding a delay made it work:

main :: IO ()
main = do
  _ <- insertData
  threadDelay 1000000
  myTweet <- queryTweet
  print myTweet



On Thu, Apr 10, 2014 at 12:41 AM, Christopher Allen <cma@bitemyapp.com> wrote:
Repro'd on GHC 7.8.1 and 7.6.3

Original code is: https://github.com/bitemyapp/bloodhound/

The code only works if I manually (out of band) run insertData (in a REPL) and then run main with the insertData invocation stripped out. If I run main with the insertData invocation included, it throws an exception (head) because the search results are empty.

The behavior is as if queryTweet was executing after insertData deleted the index, but before it inserted the new data.

The following is a stripped down example:

insertData :: IO ()
insertData = do
  let encoded = encode exampleTweet
  _ <- deleteExampleIndex
  created <- createExampleIndex
  docCreated <- indexDocument (Server "http://localhost:9200") "twitter" "tweet" exampleTweet "1"
  print "test"
  return ()
el
queryTweet :: IO (Either String Tweet)
queryTweet = do
  let queryFilter = BoolFilter (MustMatch (Term "user" "bitemyapp") False)
                    <||> IdentityFilter
  let search = Search Nothing (Just queryFilter)
  reply <- searchByIndex testServer "twitter" search
  let result = eitherDecode (responseBody reply) :: Either String (SearchResult Tweet)
  let myTweet = fmap (hitSource . head . hits . searchHits) result
  return myTweet

main :: IO ()
main = do
  _ <- insertData
  myTweet <- queryTweet
  print myTweet

further up the call chain, the http call is getting dispatched with http-conduit's gear, the result returned with the expression:

withManager $ httpLbs req

Included this in case it affects the semantics.

Can anybody help? This has stumped me for a couple hours and I couldn't get anything clueful on IRC.

Thanks to any that help.

--- Chris



_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners




_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners