Hi

...

"I would contribute to darcs if only..."

The darcs2 announcement strongly suggested that darcs would no longer be developed. This was brought up in the #ghc discussion about whether to switch.

I agree strongly with this. I would be much more likely to contribute if the project seemed active and alive. The darcs 2.0 announcement read like an obituary, and that put me off. The impression I came away with (accurate or not), was that the entire program had been rewritten, I was going to gain incompatibilities, and be using an untried/tested version which was not going to get any support. The message also seemed to be threatening me that if I didn't upgrade you would break into my house and wipe my data - instead of the usual enticement with cool new features :-) [Of course, a few emails/blog comments/interactions with Eric has shown me there is some life in the project - but I think a lot of people get jumpy when it comes to version control software] One thing that might help is splitting bits of darcs into libraries. There have been various things in darcs which are now separate libraries - ByteString and FilePath both have/had parallels in darcs. By making a separate library you get a better documented interface, a cleaner separation of concerns, and people can contribute small patches to self-contained elements, rather than a big application. You also provide additional benefits to the general community :-) Thanks Neil

trentbuck:

"Neil Mitchell" writes:

...

The darcs 2.0 announcement read like an obituary

I don't know why, but a lot of people I spoke to seemed to have that impression, and I essentially had to wave changelogs under their face to convince them that darcs was still being worked on *at all*. I had to point out that it was a *release* announcement -- how could a dead project have a new major version?

The 2.0 release should have been a triumph. That announce was a strategic error of pretty immense proportions. And as a result, it's basically impossible to justify work on the project to outsiders, or new transition to darcs on projects. And it made those who recommended darcs look foolish. The once shining "smart dvcs" brand of darcs has been trampled. To really turn things around -- which maybe could be done -- we'd need new leadership, and an aggressive change in project direction, to counter the slide. And that's a *lot* of work: full time, active project leadership, lots of public activity, clear goals, clear direction. It could make someone famous. Do we have a young turk to take up the challenge? And all this delay while the git juggernaut takes over the internet.

Perhaps a chirpy journalism major should be writing vapidly up-beat announcement posts, denying even the possibility of problems :-P

I think Eric's doing language processing, not journalism ;) -- Don

At Sun, 3 Aug 2008 12:23:21 +0100, Ashley Moran wrote:

GitHub is responsible for git's popularity. Git is so popular not because it's the best, but because it has the best Web 2.0 site. I work primarily in web development and it did occur to me to have a stab at darcshub, but I didn't know how much work would be involved in making something useful (I don't have much spare time right now), and if I'd be the only person that used it...

I think this view is probably coloured by your background in web development. I have used git for about a year now, and never visited GitHub. I'm not saying you have to like git, but it does have other features other than a snazzy web site. I do agree that adoption of development tools is driven by network effects. When I chose a DVCS to learn, I only wanted to learn one, so I chose the one that seemed like it had the most momentum. The rich get richer... Maybe the answer is to work on Darcs-git :-) David

Ashley Moran wrote:

On Aug 03, 2008, at 3:36 pm, David Bremner wrote:

...

I think this view is probably coloured by your background in web development. I have used git for about a year now, and never visited GitHub. I'm not saying you have to like git, but it does have other features other than a snazzy web site.

Hi David

I think I gave the wrong impression there. After all, I use darcs despite it not having a snazzy website! What I mean is that git usage has snowballed since GitHub was released, so people are clearly attracted to the website first, and the SCM second. It's a bit like the way Rails created thousands of Ruby programmers by association, many of them with no idea what Ruby was all about, just a vague notion that Rails could solve their problem.

I know of exactly 0 programmers that use Git because of GitHub. I have interacted with exactly 1 project that used GitHub. I know of many, many programmers that use Git. Git people are choosing Git for other reasons. I've spelled out some of the reasons I chose it before; I could bore you with URLs if you like ;-) Git is a really nice DVCS. That has nothing to do with the presence of one particular website.

I tend to very stubbornly work the other way... choose the tool I think works best with very little regard for its momentum, unless of course it clearly has none. Hence my love of darcs and recent interest in Haskell. (I'll figure it out, one day!)

Haskell has momentum, I swear!

There's also discussion on darcs-users that a Haskell implementation of Git would finally settle the "Haskell is too slow" debate. Now I think if the world is going to use git, a better implementation would be a good thing (I know a developer who got VERY frustrated trying to program against it). Personally I think the developer time would be better invested in fixing darcs bugs and improving its performance.

Yes, I am not sure the world needs a reimplemented Git. As a user, I would say, "what's the benefit?" I don't see one. That's an awful lot of work. As a developer, yes Git's internals are, shall we say, inconsistent. But what do I care? Git's interface is a shell tool, and I can use any programming language I want to work with it. I've already done so with sh, Haskell, and (ugh) Ruby. -- John

"Brandon S. Allbery KF8NH" writes:

...

"Neil Mitchell" writes:

...

...

The darcs 2.0 announcement read like an obituary

...

I don't know why, but a lot of people I spoke to seemed to have that impression, and I essentially had to wave changelogs under their face to

[...]

didn't help. All in all, it sounded an awful lot like "here's our final release, find new maintainers if you want it to continue" --- followed by an ominous silence from the darcs camp.

Well, it seems to be up to 2.0.2 now, so it's clearly too early to pass out the shovels. On the other hand, I was trying to pull down the repo, and got a 404 - that's not so good. Is it down temporarily, or is the front page incorrect? -k -- If I haven't seen further, it is by standing in the footprints of giants

Gwern Branwen writes:

I just darcs get http://darcs.net, so I would guess it was either temporary or a problem on your end.

Seems I needed a newer darcs - the one shipped with Ubuntu is 1.0.9, which appears to be too old, and it works when I build a new 2.0.2 from the tarball. (Anybody with write access to the front page who can make a note of minimum version required to 'darcs get' the repository?) -k -- If I haven't seen further, it is by standing in the footprints of giants

Ashley Moran writes:

On Aug 03, 2008, at 5:36 pm, Ketil Malde wrote:

...

Seems I needed a newer darcs - the one shipped with Ubuntu is 1.0.9, which appears to be too old, and it works when I build a new 2.0.2 from the tarball. (Anybody with write access to the front page who can make a note of minimum version required to 'darcs get' the repository?)

AFAIK, the current darcs darcs repo should be gettable with any 2.x version of darcs. Since darcs is self-hosting, you gotta expect bootstrapping issues when using older-than-the-current-release versions to checkout the repo. (Hence grabbing the source tarball.)

FYI, Iain Lane made a darcs package for 2.0.2: https://launchpad.net/~laney/+archive

You should also be able to backport it from Intrepid (Hardy+1). There may be a hardy-backports version, too, but packages.ubuntu.com won't talk to me just now.

Brandon S. Allbery KF8NH wrote:

(Hm, I feel a blog rant coming on.)

I take it you mean as the perfect example of how to kill off interest in your own project :-) I can't help thinking this was so obviously going to happen that it must have been Davids intent at the time he wrote that announcement. The only doubt I have is whether he was just in a grumpy mood at the time and now regrets this, or whether he still feels this way. To answer the OP's question, when I first looked at darcs I was quite enthusastic about it and did indeed consider hacking on it. But a few hours browsing the source code made me realise I would never be able to do this easily because of the lack of decent documentation of source. It was "literate" haskell (which I dislike anyway) and the literate commentation that I could see told me practically nothing about the code I was actually looking at. Instead it was just the markup that would eventually become the user guide (presumably). So as well as having no useful documentation (for a would be contributor) I had make the extra mental effort deliberately avoiding reading the distacting and utterly irrelevant literate commentation clutter. It would have been easier if there was no comments at all. The net result was that I couldn't even figure out what the various functions I was looking at were trying to do (and hence whether of not they might be the source of the bug or performance bottleneck or whatever). Put simply, (as someone else observed earlier) it seemed to me like it was written and organised so as to be unmaintainable by anyone other than David himself. That said, I think by normal the standards of OS projects darcs *user* documentation is very good, but for would be hackers this is not enough (decent source documentation is needed too IMO). I also think Neils idea of breaking darcs up from 1 monolithic prog to a darcs lib suite is a good idea. This would give decent haddock documentation for most of the code base and an easy way to have multiple user interfaces (gui/web/command line based). Regards -- Adrian Hey

Excerpts from Andrew Coppin's message of Sun Aug 03 04:35:32 -0500 2008:

Correct me if I'm wrong, but... I was under the impression that Darcs is a revision control system. It controls revisions.

Well Darcs already does that. So... what's to develop? It's not like it's slow or buggy. I can't actually think of any features it doesn't have that I want. So... what now?

In case that sounds really negative and critical, let me phrase it another way: Given that Darcs is more or less perfect now, what's to add?

At this point I feel it is an issue of what can be fixed rather than what can be added. The current issues with darcs right now I feel are: * Predictability. There is no such thing with darcs. If I try to get copies of the base libraries up to date in the latest GHC-head (via a ./darcs-all pull -a,) it might pull all the libraries. It might not. It might randomly stop at identifying the repository format of 'base/array.' It might get all the way to 'base/unix' and begin pulling patches before it stops. In either case, darcs never quits, I'm never sure of what state it is in during the pull and frankly I just don't know what's going on sometimes. * Usability is dodgy under high load. To date, I think GHC is the best candidate for qualifying as 'high load,' but even then I'm not sure if the comparison is quite as fair if GHC is still using an old-fashioned darcs format (i.e. darcs1-style repo.) Regardless I am still skeptical at this point of darcs ability to handle high load. * Random quirks, e.g. some file edits may unknowingly introduce an explicit dependency on patches, meanining while two patches A and B are unrelated in all forms of the word, it will still mark 'A' as dependent on 'B'. So you've basically lost the ability to unrecord flexibly, since unrecord is going to skip the patches that are marked as dependent on something else, when they really are not. Quite honestly I think darcs 2 is a fine release and a very good improvement; it fixed a lot of the older nasty quirks (exponential-time merge bug in particular) and any project considering darcs should use the new format immediately. I think for medium to small sized projects it is great. It will continue to fit those projects very well, but when push comes to shove, darcs won't cut it, I feel. To answer the question and be honest about it: I would work on darcs if I was getting paid. Darcs lost I think because of its internals, because it simply was not feasible for people to get involved and make core improvements. You basically were either David or you weren't. Other VCS's generally speaking have a lot more simple foundation: git only has only 4 base objects that you encounter every day in your workflow. These are the primitives of all operations in git, and everything else is built on top of them. Because of it, anybody with some experience in using git can probably get reasonably comfortable with the source code in a non-crazy amount of time. There is a lower barrier of entry to the source code, ideas and development process. The simpler design simply facilitates contributions where it matters, IMO. While I don't want to sound so unsupportive of the excellent work everybody involved in darcs has contributed, that is currently the way I see darcs and unless I was getting paid I would not be compelled to work on the source code as I have already moved to another system (git.) I think darcs made a landmark is usability for a DVCS, and I also think the way in which patches are managed with darcs is very nice. However, I've found git's approach sufficiently simple (to the point where my limited brain can comprehend it) and cherry-picking is a common point among version control systems these days (git add -i.) So I've found home elsewhere. Austin

On 2008 Aug 3, at 5:35, Andrew Coppin wrote:

Well Darcs already does that. So... what's to develop? It's not like it's slow or buggy. I

Oh, two more under "buggy": (a) as mentioned by others, the ghc repos often cause darcs2 to spin without doing anything. (This may secretly be the network library; strace looks an awful lot like my lambdabot when it gets stuck, but it's hard to tell because that may just be the thread scheduler.) (b) it doesn't handle the case where a file is renamed to avoid upper/ lowercase issues, on filesystems which treat them the same (Windows, OSX). It should be possible to arrange to not have both on disk at the same time. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

Andrew Coppin wrote:

Trent W. Buck wrote:

...

I don't know why, but a lot of people I spoke to seemed to have that impression, and I essentially had to wave changelogs under their face to convince them that darcs was still being worked on *at all*. I had to point out that it was a *release* announcement -- how could a dead project have a new major version?

Perhaps a chirpy journalism major should be writing vapidly up-beat announcement posts, denying even the possibility of problems :-P

Correct me if I'm wrong, but... I was under the impression that Darcs is a revision control system. It controls revisions.

Well Darcs already does that. So... what's to develop? It's not like it's slow or buggy. I can't actually think of any features it doesn't have that I want. So... what now?

Erm, actually it is. Both. Well, at least the slow bit is still quite there in 2.0, though better. The IndempotentMerge problem? I guess it sounds like this is better in 2.0, though not completely fixed. I still laugh when I remember droundy telling me something along the lines of, "No, that is NOT an infinite loop. It will finish in a couple of weeks." <grin> -- John

wren ng thornton writes:

[Bug trackers are an excellent source of tasks for active developers to use so things don't get lost, but they're awful for new developers. For someone just joining the project it's rarely clear how important a task is, how hard, or how far reaching its consequences (or whether someone's already working on it). Good trackers have fields to note these things, but the notes are engineered for active developers; the extent to which those notes are even used or accurate varies wildly from project to project. Hence, having a clear discussion about what things really are important and how much they interact with everything else is a great boon.]

Agreed. In short, shouldn't Darcs come up with sth like http://wiki.winehq.org/JanitorialProjects or http://janitor.kernelnewbies.org/ perhaps? And of course with some serious up-to-date documentation on the theory behind Darcs. AFAIK Ian Lynagh started working on one. I'd say: first be precise. Don't be afraid of abstract algebra, it's university material, quite some people actually understands it. And those can later explain the hard to grasp parts. But I never felt like diving into the bunch of hazy metaphors I found about the inner workings of Darcs, even though I was and still am interested. So I nevert felt qualified to touch anything important or assess the performance problems for example. -- Cheers, Feri.

On Aug 2, 2008, at 19:26 PM, Jason Dusek wrote:

Eric Kow wrote:

...

"I would contribute to darcs if only..."

...there were interest in binary file handling.

I'm interested in binary file handling. But what do you mean -- do you want darcs to do some kind of binary deltas and then merge them? Sounds crazy. Or do you just want it to be efficient in time and space to hurl big files around without applying patches to binaries? By the way, one of the things that intrigues me is that the best open source compression tool out there is written in Haskell: https://haskell.org/bz There might be some interesting advantages that a revision control tool could gain with the use of a good compression tool like that... Regards, Zooko

zooko wrote:

Jason Dusek wrote:

...

Eric Kow wrote:

...

"I would contribute to darcs if only..."

...there were interest in binary file handling.

...what do you mean -- do you want darcs to do some kind of binary deltas and then merge them? Sounds crazy. Or do you just want it to be efficient in time and space to hurl big files around without applying patches to binaries?

I don't see how you can be efficient in space without doing binary deltas, but I don't care for binary merging (mergelets, anyone?). -- _jsn

On 2008 Aug 3, at 13:15, Luke Palmer wrote:

On Fri, Aug 1, 2008 at 3:45 PM, Eric Kow wrote: "I would contribute to darcs if only..."

I haven't used darcs much, so it's possible that I'll be forced to start contributing by my own binding hypothetical.

I would contribute to darcs if only it had support / could have support for splitting and merging repositories. For example, I

Maybe that's what you should contribute. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

On 2008 Aug 3, at 19:16, Ben Franksen wrote:

The naive way to emulate your split feature would be to create a branch where you delete all the stuff you don't want and then maybe move the subproject to a new directory (nearer the top-level). This doesn't work, however, at least not in practice. This is because deletion of a file conflicts with a change to the same file which leads to a huge amount of conflicts each time you pull from the old combined repo. And the reason you get these conflicts is that in darcs a file always gets emptied before deletion, and this is because changing a file depends on its existence in the first place. I proposed to change this and allow changes to non-existing files, so called 'ghosts'. This has a number of interesting consequences, among them that you could delete as many files as you want and will never again get a conflict with changes to those files (that is, unless you explicitly 'resurrect' the ghost).

Unfortunately few people (and none of the core-developers) seemed to be interested :( The small thread that developed on the darcs-users list should still be available in the archives if you are interested in the details.

I would suggest that they'd be more interested if you provided code; if they have no need for your proposal they're unlikely to devote time to coding it. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allbery@kf8nh.com system administrator [openafs,heimdal,too many hats] allbery@ece.cmu.edu electrical and computer engineering, carnegie mellon university KF8NH

Ketil Malde writes:

Duncan Coutts writes:

...

It would be useful if some darcs 2 hackers, contributors could help the ghc people evaluate if darcs 2 is still in the running.

This looks like a very easy and low-investement way to get involved.

...and now I feel I've done my part :-) Anyway: I took the 'biolib' repo with 2005 patches, and made three copies, here's the timing results: darcs get ../biolib biolib-1 0.20s user 0.16s system 100% cpu 0.360 total darcs get --hashed ../biolib biolib-h 0.64s user 0.17s system 98% cpu 0.813 total darcs convert ../biolib biolib-2 0.62s user 0.18s system 7% cpu 11.101 total (The 11 seconds in the last one is me typing in "I understand the consequences of my action". The warnings are almost scary enough for me to worry about the state of my source repo, but I trust it is safe to do this and discard the new repos afterwards..) The 'convert' printed something looking like a patch: merger 0.0 ( hunk ./Bio/Alignment/AlignData.hs 5 -module Bio.EditList.AlignData ( $ +module Bio.Alignment.AlignData ( $ hunk ./Bio/Alignment/AlignData.hs 3 -{-# OPTIONS -fglasgow-exts #-} - -module Bio.EditList.AlignData ( $ +module Bio.EditList.AlignData ( ) and I'm not quite sure what this is about? Anyway, I then proceeded to run various queries (what, changes, diff), but apparently, this repo is way too small to actually show anything useful, it all completes in about 0.05-0.2 seconds. The consequences of moving to the darcs-2 format are a bit unclear to me. For instance, I'd like to keep my main (export) repo in darcs-1 format, in order to make it as accessible as possible (Ubuntu still ships with darcs-1.0.9, and that's a fairly cutting edge distribution.) Can I convert my working repos to darcs-2? Should I? How about darcs-hashed? In short, I'd like to see examples of recommended migration strategies, and associated benefits. -k -- If I haven't seen further, it is by standing in the footprints of giants

Ben Franksen writes:

...

Can I convert my working repos to darcs-2?

No. You cannot push or pull between darcs-2 format repos and darcs-1 or hashed format repos. This might not be optimal but is understandable, since the theory underlying the darcs-2 repository format is different.

So if I'm pulling from/pushing to a darcs-2-format repo, my local repo must be darcs-2 as well?

The more important problem is that conversion to darcs-2 format doesn't support multiple branches at all.

To make this perfectly clear: this is the *conversion process* only, you can still have multiple branches *after* all repos have been synced to a single conversion. Right?) Should something like this work? darcs convert main main-2 cd branch mv _darcs old_darcs cp -a ../main-2/_darcs . darcs rec -am "amalgam of changes in 'branch'" (But you'd lose your recorded history specific to the branch.) -k -- If I haven't seen further, it is by standing in the footprints of giants

On Aug 5, 2008, at 0:26 AM, Ketil Malde wrote:

The consequences of moving to the darcs-2 format are a bit unclear to me. For instance, I'd like to keep my main (export) repo in darcs-1 format, in order to make it as accessible as possible (Ubuntu still ships with darcs-1.0.9, and that's a fairly cutting edge distribution.) Can I convert my working repos to darcs-2? Should I? How about darcs-hashed? In short, I'd like to see examples of recommended migration strategies, and associated benefits.

Here's what I recommend, and what I've been doing for months, both with the large Tahoe code base -- http://allmydata.org/trac/tahoe and with numerous smaller codebases -- http://allmydata.org/trac . 1. Upgrade any executables you have to the latest stable version of darcs immediately. The new executables are fully backwards compatible -- you will not have any compatibility problems as a result of upgrading. They are also much less buggy and faster than older darcs executables. (There are at least two known issues with the darcs-2.0.2 executable which cause performance problems -- one is a issue with ghc v6.8.2 and the other is an issue with the latest version of libcurl -- v7.18.2. Hopefully the darcs core team will release darcs-2.0.3 soon to fix these two issues, but even if darcs-2.0.2 is still the current version, I recommend it over any previous version.) If your use of darcs is currently satisfying (fast enough, no bugs getting in your way), then stop at this step. You are now using the latest stable darcs executable while using old-fashioned darcs-v1 repositories. Other people who are using older versions of darcs can continue to share patches with your repositories just like always. 2. If there are problems with this situation of using darcs-2 executables and darcs-1-format repositories, then report it to the darcs-users mailing list, and then make a new hashed-format repository in parallel with your existing darcs-1-format repository, like this: darcs get --hashed ${repo} ${repo}-hashedformat Now people who have new darcs-v2 can push and pull to the - hashedformat repository and people who have darcs-v1 can continue to push and pull to the original repository. You can transfer patches between those repositories with normal darcs push and pull. For those projects of mine which have a high rate of patches I have gone ahead and added post-apply hooks to automatically do a "push --all" to the other one, like this: echo "apply posthook darcs push --all --repodir=/home/source/darcs/ tahoe/${repo}-hashedformat /home/source/darcs/tahoe/${repo}" >> $ {repo}-hashedformat/_darcs/prefs/defaults echo "apply run-posthook" >> ${repo}-hashedformat/_darcs/prefs/defaults echo "apply posthook darcs push --all --repodir=/home/source/darcs/ tahoe/${repo} /home/source/darcs/tahoe/${repo}-hashedformat" >> $ {repo}/_darcs/prefs/defaults echo "apply run-posthook" >> ${repo}/_darcs/prefs/defaults (Note that this is mutually recursive, but it terminates because the post-apply hook doesn't trigger in the case that zero patches were pushed.) If your use of darcs is currently satisfying then stop at this step. You are now using the latest stable darcs executable, and maintaining two darcs repositories -- an old-fashioned darcs-1-format repository for your darcs-1-executable-using friends and a hashed-format repository for your darcs-2-executable using friends. Your post- apply hooks cause these two repositories to get automatically synced with each other. 3. If there are problems with this situation of using darcs-2 executables and a pair of repos -- one of which is darcs-1-format repo for the use of darcs-1-executable-using users and the other of which is a hashed-format repo for the use of darcs-2-executable-using users -- then report it to the darcs-users mailing list and await further instructions. Note that these instructions do not involve any use of darcs-2-format repositories. Those are only for projects which (a) have no darcs-1- executable-using users, and (b) have no extant branches stored in darcs-1-format or hashed-format repositories. Personally step 2 has satisfied all my needs so I use the step 2 strategy for all my publicly shared repositories, although I do tend to create new private repositories with darcs-2-format. If someone wants to adapt this message to the darcs wiki I would be grateful. Regards, Zooko http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month

Could darcs 2 performance be improved by making use of the order of patches in the reference repo, to identify reference versions and reign in exponential permutation issues? In other words, all repos are equal, all patches are equal, but once a patch has made the roundtrip through the reference repo, it i

s more than equal - there would be unlocated and located (wrt to the reference repo) patches. Claus