Squashing, rebasing, and conflicts

Question
1

Hi there, I am brand new to Pijul as of the last 48 hours… I’ve read all the docs, a bunch of forum posts, watched some vids on YouTube, and have been experimenting.

My big caveat: I’ve been using git pretty much daily for 16+ years. Please be patient with me as I try to wrap my head around the Pijul model :slight_smile:

There’s a code workflow that’s fairly common, that git makes quite painful. I’d like to know if I can accomplish the desired result with Pijul.

In short, it is:

  • make a bunch of commits on a branch
  • squash some of the intermediate commits

It’s not a problem if I’m the person doing it, and it’s all in my local workspace. The problem arises when someone else creates a messy history, and I want to clean it up, but keep some later commits.

Similarly, it can occur if two people independently create the same commit.

(and now I will switch to Pijul terminology)

For a slightly more specific scenario:

  1. I record a good change
  2. Joe records two messy changes, but good state
  3. I record that same good state as a good change
  4. Joe records a new change on top of his messy changes

I want Joe to replace his messy changes with my good one, and maintain his new work (i.e. rebase).

Here’s a script that illustrates the scenario (pijul 1.0.0-beta.9):

#!/bin/sh
set -eu

rm -rf demo
pijul init demo
cd demo

echo "line 1" >> myfile
pijul add myfile
pijul record -a -m "add line 1"

pijul fork joe-fork

echo "line 2" >> myfile
pijul record -a -m "add line 2 in joe-fork" --channel joe-fork
echo "line 3" >> myfile
pijul record -a -m "add line 3 in joe-fork" --channel joe-fork

pijul record -a -m "add lines 2,3 in main"

echo "line 4" >> myfile
pijul record -a -m "add line 4 in joe-fork" --channel joe-fork

pijul pull -a --to-channel joe-fork .

pijul reset .
pijul channel switch joe-fork
cat myfile

This produces the following output:

Tracked 1 path(s)
Hash: GJDJIVG6DFXJQ65CEBLRWKZDVUMLJSXGCC2FT7X4ZCP36SBVLM5QC
Hash: BMXON7WXENSKWWC2AB53RAVIOA6G5C5CY7TTZBN2PUUA5SUODYSQC
Hash: BNN3LIPZ5QLOS6MFN4GDZ246MAF7HBYRT72YIGGGXNUE7DSZVL5QC
Hash: UVSEJM7N2ICMB35CQ523RAS5UNNHHR5PROIY7I6UKFHUJ2HTOV7QC
Hash: 26HQBCNUR2WBWM2ZAHOCDTYRV46FJUCTTF6YJ3PKHMOTEEXSNR7AC
Downloading changes  [==================================================] 1/1 [00:00:00]
Applying changes     [==================================================] 1/1 [00:00:00]
Downloading changes  [==================================================] 1/1 [00:00:00]
Completing changes... done!
Outputting repository... done!
Reset given paths to last recorded change
Outputting repository... done!
There were conflicts:

  - Order conflict in "myfile" starting on line 2
Reset given paths to last recorded change
line 1
>>>>>>> 1 [BMXON7WX add line 2 in joe-fork]
line 2
line 3
line 4
======= 1 [UVSEJM7N add lines 2,3 in main]
line 2
line 3
<<<<<<< 1

The question

How do we get joe-fork into this nice, clean set of changes?

  • “add line 1”
  • “add lines 2,3 in main”
  • “add line 4 in joe-fork”
2

Related to this is what to me looks like an odd diff after I resolve the conflict. Here, the diff shows it removing lines 2 and 3. This kind of makes sense in that the lines are being removed from the conflict markers - but if I were to ask “do I want to pull a change that removes these two lines?” the answer would be no. But of course the file is in the desired state - the lines have only been removed from the conflict markers.

I don’t have a specific question around this, other than how should I be thinking differently to make sense of this change?

#!/bin/sh
set -eu

rm -rf demo
pijul init demo
cd demo

echo "line 1" >> myfile
pijul add myfile
pijul record -a -m "add line 1"

pijul fork joe-fork

echo "line 2" >> myfile
pijul record -a -m "add line 2 in joe-fork" --channel joe-fork
echo "line 3" >> myfile
pijul record -a -m "add line 3 in joe-fork" --channel joe-fork

pijul record -a -m "add lines 2,3 in main"

echo "line 4" >> myfile
pijul record -a -m "add line 4 in joe-fork" --channel joe-fork

cp myfile /tmp/myfile.good

pijul pull -a --to-channel joe-fork .

pijul reset .
pijul channel switch joe-fork

cp /tmp/myfile.good myfile
pijul diff

output:

Repository created at demo
Tracked 1 path(s)
Hash: HRGN52DK63B7NL6REJDTQWAEFOSTVJQKOVHPE4CLIOQ5XJF34MEAC
Hash: C4N6DK2N4HLMZ3FJWFKJMGX5EU4767ZVR374JAAYDXOSLTTXI5RQC
Hash: MZQKIT7SEZSHGUMQVZOEXDLTQOJ32J5XWQBH5IZJA36ZGQ4WIS7QC
Hash: U7YO5LLT2EEATBNBMVW4WIM4P6C3JJKGMMPTA7WKIJ2MOYFZZ7NAC
Hash: DVMMLJQZ67AWOLIOV2MFGWUEA5FPKWJMEICFREVK7ZOQMVAV2WIQC
Reset given paths to last recorded change
0m31m
There were conflicts:

0m  - Order conflict in "myfile" starting on line 2
Reset given paths to last recorded change
message = ''
timestamp = '2024-10-10T22:54:20.017918473Z'
authors = []

# Dependencies
[2] DVMMLJQZ67AWOLIOV2MFGWUEA5FPKWJMEICFREVK7ZOQMVAV2WIQC # add line 4 in joe-fork
[3] U7YO5LLT2EEATBNBMVW4WIM4P6C3JJKGMMPTA7WKIJ2MOYFZZ7NAC # add lines 2,3 in main
[4]+HRGN52DK63B7NL6REJDTQWAEFOSTVJQKOVHPE4CLIOQ5XJF34MEAC # add line 1
[*] HRGN52DK63B7NL6REJDTQWAEFOSTVJQKOVHPE4CLIOQ5XJF34MEAC # add line 1
[*] C4N6DK2N4HLMZ3FJWFKJMGX5EU4767ZVR374JAAYDXOSLTTXI5RQC # add line 2 in joe-fork

# Hunks

1. Edit in "myfile":5 4.1 "UTF-8"
B:BD 4.9 -> 3.1:15/3
- line 2
- line 3

2. Solving an order conflict in "myfile":5 4.1 "UTF-8"
  up 2.8, new 2:2, down
3

I’ve been experimenting more, and I think I’ve identified a source of my misunderstanding…

I had been working off the belief that Pijul operates on sets of diffs - and that applying a change would apply the diff from one channel to another.

It seems that Pijul actually operates on sets of patches, identified by a hash. Applying a patch includes all the dependencies. Changing a patch’s hash (e.g. by changing the message) will result in a conflict on next pull:

#!/bin/sh
set -eu

rm -rf demo
pijul init demo
cd demo

echo "line 1" >> myfile
pijul add myfile
pijul record -a -m "line 1 - main"
echo "line 2" >> myfile
pijul record -a -m "line 2 - main"

pijul fork mychannel
pijul reset --channel mychannel
pijul record -a --amend -m "new message"
pijul pull -a --from-channel main .

produces:

Repository created at demo
Tracked 1 path(s)
Hash: 66MN2OJKHUDTNFDZBMAR7Q3FEA6EPSPUC2I6QMJTUZSGHFC2NGHAC
Hash: JU2BWFBDMMG3CANG3SVFGW4LWSNUJEIDHJLRTD5CV2O3VU3MPOEQC
Outputting repository... done!                                                                                                                                                                                                 Reset given paths to last recorded change
Hash: POEPJXQF5JPP7SLWMGHSWKHEPHATLU2QPPKDCQHCOEQBI6V4VJ3QC
Downloading changes  [==================================================] 1/1 [00:00:00]                                                                                                                                       Applying changes     [==================================================] 1/1 [00:00:00]                                                                                                                                       Downloading changes  [==================================================] 1/1 [00:00:00]
Completing changes... done!                                                                                                                                                                                                    Outputting repository..
There were conflicts:

  - Order conflict in "myfile" starting on line 2
Outputting repository... done!

So I think the solution to my original question is to unrecord the first conflicting messy change, and all its dependents. Then diff against the desired tree and produce new clean commits.

Here’s what that looks like:

#!/bin/sh
set -eu

rm -rf demo
pijul init demo
cd demo

echo "line 1" >> myfile
pijul add myfile
pijul record -a -m "line 1 - main"

pijul fork joe-fork
pijul reset --channel joe-fork
echo "line 2" >> myfile
pijul record -a -m "line 2 - joe"
echo "line 3" >> myfile
pijul record -a -m "line 3 - joe"

pijul reset --channel main
echo "line 2" >> myfile
echo "line 3" >> myfile
pijul record -a -m "lines 2,3 - main"

pijul reset --channel joe-fork
echo "line 4" >> myfile
pijul record -a -m "(temp) add line 4 in joe-fork"

pijul fork joe-good-tree

pijul reset --channel joe-fork
pijul pull -a --from-channel main .

hash=$(cat myfile | grep '>>>>>>>' | cut -w -f 3 | sed 's/^\[//')
pijul dependents ${hash} | xargs pijul unrecord --reset

pijul reset --channel joe-good-tree
(echo "message = 'line 4 - joe'"; pijul diff --channel joe-fork | tail -n +2) | pijul apply --channel joe-fork
pijul reset --channel joe-fork
pijul log
cat myfile

which produces:

Repository created at demo
Tracked 1 path(s)
Hash: YZZVLCZCYYS4LEOS5M6OKXRZH66MPQTKY3QOXL2XWPH5R3Z4LGQAC
Outputting repository... done!                                                                                                                                                                                                 Reset given paths to last recorded change
Hash: UBLMLICDOAZKV7MUDZ4BUTR6EGMDZ4WK6X2UIBRUT3HTXSIYPFBAC
Hash: COUENKPG7F3PRNMBVMUCBATR2GRBGZ3FUKRM7IXYB25SM6NGFJNAC
Outputting repository... done!                                                                                                                                                                                                 Reset given paths to last recorded change
Hash: 3HP5EQZROQAQ7Q3F2NLKBINGP6XBN4BBLVFWY4XLJBR27ZQHKK5AC
Outputting repository... done!                                                                                                                                                                                                 Reset given paths to last recorded change
Hash: 5N6I4JLVVYI5GFMAPVN7EIJ4FMOMEB7JHASGPOAB3MEMVKJNESQAC
Reset repository to last recorded change
Downloading changes  [==================================================] 1/1 [00:00:00]                                                                                                                                       Applying changes     [==================================================] 1/1 [00:00:00]                                                                                                                                       Downloading changes  [==================================================] 1/1 [00:00:00]
Completing changes... done!                                                                                                                                                                                                    Outputting repository..
There were conflicts:

  - Order conflict in "myfile" starting on line 2
Outputting repository... done!                                                                                                                                                                                                 Outputting repository... done!                                                                                                                                                                                                 Reset given paths to last recorded change
Outputting repository... done!                                                                                                                                                                                                 Reset given paths to last recorded change
Change 2577AXBYDY435CFRFJKYSRURREJT5TI5KZKX4DMY5AECPP52L62AC
Author:
Date: Fri, 11 Oct 2024 03:47:40 +0000

    line 4 - joe

Change 3HP5EQZROQAQ7Q3F2NLKBINGP6XBN4BBLVFWY4XLJBR27ZQHKK5AC
Author:
Date: Fri, 11 Oct 2024 03:47:40 +0000

    lines 2,3 - main

Change YZZVLCZCYYS4LEOS5M6OKXRZH66MPQTKY3QOXL2XWPH5R3Z4LGQAC
Author:
Date: Fri, 11 Oct 2024 03:47:40 +0000

    line 1 - main

Change UVMVFUFNDW5QCRHZ7TQ4ZGTREDZQL32DBHBNTUIRFNQCWRFPHH6AC
Author:
Date: Fri, 11 Oct 2024 03:47:40 +0000



line 1
line 2
line 3
line 4

One consequence of this is that if joe-fork has made progress across a few features, it will require re-creating each of the new patches. I suppose then an approach would be to create a desired end state (joe-good-tree channel in this case), and then pull patches into a new feature-specific channel. Diff the feature channel against joe-work to produce the new patches, repeat for each feature.

That’s what I’ve come up with so far… I’d definitely love to hear better, more Pijul-oriented approaches.

4

Unfortunately I don’t really have a good answer for this, but your posts did help me understand pijul a little bit more. I think you ran into a similar thing that I ran into when I playing around with pijul and posted this question: Is there a way of telling if changes, saved in text files, will conflict? (although the question I ended up asking was different).

Basically, you sometimes run into a situation where you don’t actually have a conflict, but pijul thinks that you do. You, the programmer, know that the resulting file is exactly the same, but pijul thinks that something needs to be reconciled, even though the diffs produce the same result.

Your comment about pijul using “patches identified by hashes” instead of just diffs is good to know.

5

It actually makes sense that it would conflict, and indeed git and other VCSes that I know of work this way.

Imagine a different purpose for the text file: a transaction log. Imagine it’s the worst designed transaction log ever, with no unique identifiers, timestamps, anything. Every line is an amount, and nothing else.

You and I each enter a transaction of $100. We merge our branches, VCS sees that we made the same change, and “helpfully” throws one away. We’re now out $100!

So we have to have the VCS tell us we’re not in agreement on a shared state, and come to an agreement.

The difference is, to my understanding, Pijul does a better job of remembering the agreement than git does. In fact, git doesn’t record the agreement in the commit history at all. Instead it remembers it locally with git rerere, so that if I perform a similar rebase on a different machine, I get to go through the whole process again.

So assuming that we continue on from the new shared agreement, how do we base our old work on it?

The answer is (I believe) we redo all the work. git actually does this too, it’s just so central to the usage that it automates it (git rebase).

My hope was that pijul offers a better solution, where you can rebase work and record the nature of the rebase. I suspect that the model permits it, but it’s not implemented yet.

6

One interesting question is, what contributes to a hash?

In git, it’s everything - the patch content, the parent, the message, author, timestamp.

Pijul follows a similar approach from what I can tell. Amend a patch message and you have a new hash - and thus divergent histories. This actually surprised me. I thought it would be more like Fossil, where commits have metadata. You can amend the message and assign tags after making a commit. It’s really useful.

1 Like