[Suggestion] Reposurgeon-like tool for Pijul? (Re: GCC still uses SVN)

no_identd · February 17, 2019, 11:23pm

So, brought on by this news item from December, about the progress of GCC transitioning from SVN to git via Eric S. Raymond’s SVN-to-git Tool Reposurgeon:

https://www.phoronix.com/scan.php?page=news_item&px=GCC-Reposurgeon-Py-To-Go-90

And this rather jesting comment in the discussions about it:

I decided to check if any tools for converting from SVN to pijul already exist, because I had a few–interrelated–ideas:

Operating on the GCC repo would probably serve as the strongest benchmark you could find: If pijul works fast & error free when working on the gcc repo, it will almost certainly do so with everything else
Converting from SVN to pijul might go faster than converting from SVN to git (Albeit, perhaps not, see this mail from Eric S. Raymond in October on some of the implementation language specific issues he encountered while porting reposurgeon from Python to Go - albeit perhaps that just tells us that one oughn’t implement a similar tool for SVN-to-pijul in either of these languages: https://gcc.gnu.org/ml/gcc/2018-10/msg00001.html)
Converting from SVN to pijul might introduce fewer issues than converting from SVN to git (See this older news item: https://www.phoronix.com/scan.php?page=news_item&px=GCC-Git-Complexities and the related GCC mailing list thread from July here: https://gcc.gnu.org/ml/gcc/2018-07/threads.html#00322)
Building on whatever Eric S. Raymond’s learned while implementing Reposurgeon not once but TWICE might provide some insights on how to best implement an SVN to pijul converter
Perhaps converting from SVN to pijul to git could work faster & with fewer issues than Reposurgeon’s approach.

The old python version of Reposurgeon is here:

gitlab.com

esr/reposurgeon/blob/master/reposurgeon

#!/usr/bin/env pypy
#
# reposurgeon - a repository surgeon.
#
# By ESR, October 2010.
#
# Requires Python 2.7.2 or newer.  Also runs under 3.x - preserve this property!
#
# SPDX-License-Identifier: BSD-2-Clause
from __future__ import print_function

import sys, os, cmd, tempfile, subprocess, glob, hashlib, cProfile
import re, sre_constants, signal, shutil, copy, shlex, collections, uuid
import cgi, bz2, codecs, time, calendar, unittest, itertools, operator
import functools, filecmp, datetime, difflib, tarfile, stat
import email.message, email.parser
import heapq
import io

# This import only works on Unixes.  The intention is to enable

This file has been truncated. show original

The Go version is here:

gitlab.com

esr/reposurgeon/blob/master/src/goreposurgeon/goreposurgeon.go

package main

// This code is intended to be hackable to support for special-purpose or
// custom operations, though it's even better if you can come up with a new
// surgical primitive general enough to ship with the stock version.  For
// either case, here's a guide to the architecture.
//
// The core classes are largely about deserializing and reserializing
// import streams.  In between these two operations the repo state
// lives in a fairly simple object, Repository. The main part of
// Repository is just a list of objects implementing the Event
// interface - Commits, Blobs, Tags, Resets, and Passthroughs.
// These are straightforward representations of the command types in
// an import stream, with Passthrough as a way of losslessly conveying
// lines the parser does not recognize.
//
//  +-------------+    +---------+    +-------------+
//  | Deserialize |--->| Operate |--->| Reserialize |
//  +-------------+    +---------+    +-------------+
//

This file has been truncated. show original

Of course, one could instead also try and build on Reposurgeon instead. Not sure which approach would seem better.

no_identd · February 17, 2019, 11:25pm

Also, ESR has some tools besides Reposurgeon that might prove useful for git to pijul conversions:

no_identd · February 17, 2019, 11:36pm

Here’s some more background detail which I just found:
https://groups.google.com/forum/#!topic/golang-nuts/WstriKt2jTA

pmeunier · February 23, 2019, 11:14pm

Sounds really cool! However, I don’t think Pijul is quite ready to operate on a repository the size of GCC, the main blockers at the moment being that:

Pijul hasn’t spent enough time without important bugs in libpijul. It was almost there three months ago, but I decided for a major refactoring in the diff algorithm, and…
We’re still a bit heavy on disk space. I have lots of ideas to reduce this, and little time to implement everything I’d like to do.

Topic		Replies	Views
Pijul is super slow when compared to git Development	2	1269	August 19, 2019
Adventures in converting: git-pijul Community	10	1322	August 23, 2020
Pijul 0.12 released! Announcements	5	3296	April 30, 2019
Random Talks and Thoughts Development	19	2386	December 24, 2019
Using Pijul for a game editor Development	15	1813	February 25, 2023

[Suggestion] Reposurgeon-like tool for Pijul? (Re: GCC still uses SVN)

Related Topics