Another repo issue

Feel free to change the title because i dont know what it SHOULD be called.

I ran apui eltrans and it kind of showed a mess. For the purposes of this post (to keep it simple) I'm uploading the output of simply apu.

Why does this stuff always happen to me...

https://pastebin.com/PhYM1LXv

(EDIT: I just realized, the paste doesnt include the Hit: and Ign: and Get: of apu. Here it is right here (because i cant edit pastes that i did as a guest)):

~ ❯❯❯ apu
[sudo] password for thetechrobo:
Hit:1 https://repo.skype.com/deb stable InRelease
Hit:2 https://deb.opera.com/opera-stable stable InRelease
Hit:3 http://deb.debian.org/debian buster InRelease
Hit:4 http://deb.debian.org/debian-security buster/updates InRelease
Hit:5 http://repo.buster.elive.elivecd.org buster InRelease
Hit:6 http://deb.debian.org/debian buster-backports InRelease
Hit:7 http://www.deb-multimedia.org buster InRelease
Ign:8 http://linux.dropbox.com/debian buster InRelease
Get:9 http://linux.dropbox.com/debian buster Release [6,600 B]
Fetched 6,600 B in 4s (1,502 B/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done

That Fetched 6,600 B never changes -- the speed and time might, but its always the same amount of data...

And this is a fresh install --done last week, but it didnt happen before...

So what happens if you run "apug" ?
That should change things after dropbox has been updated... i.e there should be no data if everything is up-to-date. :smiley14:

Theyre just warnings, so apug continues.

This may be :offtopic: but its not a bug; just a question: can these packages be downgraded by apug safely? I HAVENT modified the default repos other than adding dropbox, opera, steam, and (maybe) a few others??? from the installer. I rarely use dropbox but with a 250GB SSD, whats to lose? :madness:

dolphin-compat-notinstalled konqueror-compat-notinstalled
  nautilus-compat-notinstalled pcmanfm-compat-notinstalled smb-admin

I will run it after I receive verification that this is normal. Tho it'll take a few hours (510 MB of archives! :open_mouth:)

Yes, the so called downgrades generally refer to the github numbering after the major numbers. The repo data base has some issues with the letters in the numbering scheme. If in doubt compare the major version numbers.

Speed!
Dropbox compares hashes to check if something has changed... i.e your system will have to compute the hash of the locally stored files every time you connect.
If that is a large trove of data it will eat CPU and RAM until it's done. I've seen it almost freeze computers at start-up, quite often.
It's the first thing I ask if people complain about temporary non-responsive machines.:smile_cat:

You can stop/pause the process if wanted (Ctrl + c). It'll keep the already downloaded packages in cache and on subsequent restart (of apug) continue where it left off

I figured that out a while ago, its really cool :slight_smile:

I never have dropbox OPEN, its just installed.
And I never reboot, I only hibernate.

So Dropbox doesnt have a chance against me :rofl2:

Ok, good to know :slight_smile:

Welp, didnt change anything, still EXACTLY the same...

W: Skipping acquire of configured file 'buster/games/binary-amd64/Packages' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/binary-i386/Packages' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/i18n/Translation-en_US' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/i18n/Translation-en' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/dep11/Components-amd64.yml' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/dep11/icons-48x48.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/dep11/icons-64x64.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/dep11/icons-64x64@2.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/Contents-i386' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'buster/games/Contents-amd64' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'buster/games' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/binary-amd64/Packages' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/binary-i386/Packages' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/i18n/Translation-en_US' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/i18n/Translation-en' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/dep11/Components-amd64.yml' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/dep11/icons-48x48.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/dep11/icons-64x64.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/dep11/icons-64x64@2.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/Contents-amd64' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'amd64/Contents-i386' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'amd64' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/binary-amd64/Packages' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/binary-i386/Packages' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/i18n/Translation-en' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/i18n/Translation-en_US' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/dep11/Components-amd64.yml' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/dep11/icons-48x48.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/dep11/icons-64x64.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/dep11/icons-64x64@2.tar' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/Contents-i386' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)
W: Skipping acquire of configured file 'Packages/Contents-amd64' as repository 'http://repo.buster.elive.elivecd.org buster InRelease' doesn't have the component 'Packages' (component misspelt in sources.list?)

???????????

HAHA IT WASN'T MY FAULT AT ALL :madness:

Thank you to this answer, you're an absolute legend

Look at this terminal output:

~ ❯❯❯ file /etc/apt/sources.list.d/*
/etc/apt/sources.list.d/aaa-elive.list:                             data
/etc/apt/sources.list.d/aaa-elive.list.save:                        ASCII text
/etc/apt/sources.list.d/debian.list:                                data
/etc/apt/sources.list.d/debian.list.save:                           ASCII text
/etc/apt/sources.list.d/dropbox.list:                               data
/etc/apt/sources.list.d/dropbox.list.save:                          ASCII text
/etc/apt/sources.list.d/ggg-marillat.list:                          ASCII text
/etc/apt/sources.list.d/ggg-marillat.list.save:                     ASCII text
/etc/apt/sources.list.d/openshot_developers-ubuntu-ppa-groovy.list: ASCII text
/etc/apt/sources.list.d/opera-stable.list:                          ASCII text
/etc/apt/sources.list.d/opera-stable.list.save:                     ASCII text
/etc/apt/sources.list.d/skype-stable.list:                          ASCII text
/etc/apt/sources.list.d/skype-stable.list.save:                     ASCII text

You'll notice that three files are labeled as "data" and the rest "ASCII text". Also if you check them in nano the files just have ^@ over and over.

It turns out that the files were corrupt! So what I did was run the following commands:

sudo su
root /e/a/sources.list.d ❯❯❯ mv aaa-elive.list aaa-elive.list.BINARY
root /e/a/sources.list.d ❯❯❯ mv debian.list debian.list.BINARY
root /e/a/sources.list.d ❯❯❯ mv dropbox.list dropbox.list.BINARY
root /e/a/sources.list.d ❯❯❯ cp dropbox.list.save dropbox.list
root /e/a/sources.list.d ❯❯❯ cp aaa-elive.list.save aaa-elive.list
root /e/a/sources.list.d ❯❯❯ cp debian.list.save debian.list

What this does is takes advantage of the <NAME>.list.save files that are just backups of the <NAME>.list files. They weren't corrupt if you look at the terminal output. It renames the corrupt files to <NAME>.list.BINARY and copies the <NAME>.list.save into <NAME>.list files.

Then, because I have no idea what else went corrupt, I ran sudo touch /forcefsck because apparently it forces fsck to run on the next boot (I rarely restart the computer, I use hibernation but I have been testing 3.0.6 in LIVE MODE as you guys suggested, sadly it doesnt work with the macbook because its not uefi, but it's an amazing work :+1:

And now it's happily downloading the things it wouldnt before :blush:

Just for the ones who care, here's apug's current output as of now: apug output - Pastebin.com

And EDIT: Dropbox doesnt seem to keep downloading the same 6,600B into cache, presumably because apt isn't stopping the process due to the fatal error)

(EDIT 2: If you've noticed, I've just learned the <sub> and <sup> html tags, and i can do things like

Uhm...(dead link) is <sub>
e = mc2 is <sup>
:offtopic:
ye i dont care
:idea:

OK, a corrupted .list file. :face_with_head_bandage:
On a side note:
Sort of remembering you had these kind of issues before (or are they still the same?), it might be worthwhile looking into why these corruptions exist in the first place.
Do you have a bad/dying disk or such? :thinking:

Which kind of issue?

This isn't the same drive as the one from earlier (the first repo mess); it's a completely different computer!

And when I shut down the computer, I swear I saw something like <SOMETHING>: disk is corrupt for a second before plymouth kicked in.

:baffled:

Look if you can find errors saved one of the files in /var/log/ like "messages" or "syslog". I'm not on Debian now (Manjaro) so cannot point you more specifically.

Ran cat * | grep -i "corrupt", and got this output

cat: apache2: Is a directory
cat: apt: Is a directory
Jul 14 20:12:03 localhost sudo:     root : TTY=pts/0 ; PWD=/etc/apt/CORRUPT ; USER=root ; COMMAND=/usr/local/bin/apt update
Jul 14 20:12:11 localhost sudo:     root : TTY=pts/0 ; PWD=/etc/apt/CORRUPT ; USER=root ; COMMAND=/usr/local/bin/apt -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confnew dist-upgrade
cat: boot.log: Permission denied
cat: btmp: Permission denied
cat: cups: Is a directory
cat: fsck: Is a directory
cat: hp: Is a directory
Binary file (standard input) matches
cat: lightdm: Permission denied
cat: ntpstats: Is a directory
cat: openvpn: Is a directory
cat: private: Permission denied
cat: samba: Is a directory

So I assume my eyes played tricks, which is no surprise...

That way you will miss out on sub-directories that contain logs too.
You might want to use "find |" grep log" for log-files first and grep that output for errors...i.e:
grep -i error $(find /var/log/ |grep log)

Why you have .save files on the sources lists ? (which software did that?)

If this is not an elive issue, maybe the thread should be removed? :thinking:

apt automatically backs up .list files AFAIK

Grepping for errors was way too much output, namely a single LONG line (i think) that had multiple instances of error. (An apt log from long ago when I was installing either Pitivi or openshot. -- included things like parse-better-error and npm-errors or something)

Here's the output of grep -i corrupt $(find /var/log/ |grep log):

Corrupt filesystem on Pastebin

So I suppose it's not my ssd...it's ext4 :madness:

I can't really run sudo fsck -fy because my disk is mounted...the only liveUSB i have is for 3.0 (and I tried that already, it said to get a new version of e2fsck)...

Considering ext4 resides on your SSD, suppose again.

I see it's sda3 that has the corrupted files. If ti's not your / (base system) you can go into single mode to run fsck with "sudo telinit 1".
I suspect those errors will keep coming at you if it's a faulty ssd but there is hope as it looks to be a single inode.

oops, ye that makes sense

That's what it is :neutral_face:

All I can think of doing is running a special option in fsck that will only READ it for errors, similar to Windoze's chkdsk. That way it can stay mounted. yes there might be false errors but...at least I'd know

then get a linux rescue-cd iso from somewhere and run that from some old small flashdisk or CD.

I advise:
https://www.system-rescue-cd.org/

It's small enough for a CD.

They've even got a page dedicated on how to get it on your SSD so you can boot into it from grub:
https://www.system-rescue-cd.org/manual/Installing_SystemRescueCd_on_the_disk/