[PATCH 3 of 4 v2] changegroup3: add empty chunk separating directories and files

[Gregory Szorc]

`hg help internals.changegroups` or view it from `hg serve`.

On Fri, Jan 15, 2016 at 10:02 AM, Martin von Zweigbergk <
martinvonz at google.com> wrote:

> Sure, that sounds reasonable. I found the file
> (mercurial/help/internals/changegroups.txt), but how do I view it's
> rendered form? "hg help changegroups" did not work.
>
> On Fri, Jan 15, 2016 at 9:54 AM Gregory Szorc <gregory.szorc at gmail.com>
> wrote:
>
>> Can we get a follow up to update the "internals" docs for cg3? Sorry I
>> forgot to mention this in the first iteration.
>>
>> > On Jan 13, 2016, at 10:09, Martin von Zweigbergk <martinvonz at google.com>
>> wrote:
>> >
>> > # HG changeset patch
>> > # User Martin von Zweigbergk <martinvonz at google.com>
>> > # Date 1452553831 28800
>> > #      Mon Jan 11 15:10:31 2016 -0800
>> > # Node ID 12f845942d02e226575a9185cda3b2ede378fe46
>> > # Parent  7d6bd9f7dfefe6a55f6569a5b977a1fad6ac4021
>> > changegroup3: add empty chunk separating directories and files
>> >
>> > Remotefilelog overrides changegroup._addchangegroupfiles(), assuming
>> > it is about files, which seems like a natural assumption. However, in
>> > changegroup3, directory manifests are sent in the files section of the
>> > changegroup. These naturally make remotefilelog unhappy.
>> >
>> > The fact that the directories are not separated from the files
>> > (although they do come before the files) also makes server.validate
>> > harder to implement. Since we read one chunk at a time from the steam,
>> > once we have found a file (non-directory) entry in the stream, we
>> > would have to push the read data back into the stream, or otherwise
>> > refactor the code. It will be easier if we add an empty chunk after
>> > all directory manifests.
>> >
>> > This change adds that empty chunk, although we don't yet take
>> > advantage of it on the reading side. We will soon move the tree
>> > manifest stuff out of _addchangegroupfiles() and into
>> > _unpackmanifests().
>> >
>> > diff --git a/mercurial/bundlerepo.py b/mercurial/bundlerepo.py
>> > --- a/mercurial/bundlerepo.py
>> > +++ b/mercurial/bundlerepo.py
>> > @@ -329,6 +329,10 @@
>> >         # consume the header if it exists
>> >         self.bundle.manifestheader()
>> >         m = bundlemanifest(self.svfs, self.bundle, self.changelog.rev)
>> > +        # XXX: hack to work with changegroup3, but we still don't
>> handle
>> > +        # tree manifests correctly
>> > +        if self.bundle.version == "03":
>> > +            self.bundle.filelogheader()
>> >         self.filestart = self.bundle.tell()
>> >         return m
>> >
>> > diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
>> > --- a/mercurial/changegroup.py
>> > +++ b/mercurial/changegroup.py
>> > @@ -506,8 +506,8 @@
>> >     """Unpacker for cg3 streams.
>> >
>> >     cg3 streams add support for exchanging treemanifests and revlog
>> > -    flags, so the only changes from cg2 are the delta header and
>> > -    version number.
>> > +    flags. It adds the revlog flags to the delta header and an empty
>> chunk
>> > +    separating manifests and files.
>> >     """
>> >     deltaheader = _CHANGEGROUPV3_DELTA_HEADER
>> >     deltaheadersize = struct.calcsize(deltaheader)
>> > @@ -909,6 +909,7 @@
>> >             yield self.fileheader(name)
>> >             for chunk in self.group(nodes, dirlog(name), nodes.get):
>> >                 yield chunk
>> > +        yield self.close()
>> >
>> >     def builddeltaheader(self, node, p1n, p2n, basenode, linknode,
>> flags):
>> >         return struct.pack(
>> > @@ -917,7 +918,7 @@
>> > _packermap = {'01': (cg1packer, cg1unpacker),
>> >              # cg2 adds support for exchanging generaldelta
>> >              '02': (cg2packer, cg2unpacker),
>> > -             # cg3 adds support for exchanging treemanifests
>> > +             # cg3 adds support for exchanging revlog flags and
>> treemanifests
>> >              '03': (cg3packer, cg3unpacker),
>> > }
>> >
>> > @@ -1054,9 +1055,13 @@
>> > def _addchangegroupfiles(repo, source, revmap, trp, pr, needfiles):
>> >     revisions = 0
>> >     files = 0
>> > +    submfsdone = False
>> >     while True:
>> >         chunkdata = source.filelogheader()
>> >         if not chunkdata:
>> > +            if source.version == "03" and not submfsdone:
>> > +                submfsdone = True
>> > +                continue
>> >             break
>> >         f = chunkdata["filename"]
>> >         repo.ui.debug("adding %s revisions\n" % f)
>> > _______________________________________________
>> > Mercurial-devel mailing list
>> > Mercurial-devel at selenic.com
>> > https://selenic.com/mailman/listinfo/mercurial-devel
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mercurial-scm.org/pipermail/mercurial-devel/attachments/20160115/6571b4f2/attachment-0002.html>