Computer block by high ram usage (SWAP saturated)

What about to connect an additional fast drive
( at least via USB 2, better 3)
and define it as cache disk
(via the options in 'gnome-disks') ?

I don't think the problem is the amount of swap, the problem is when it gets to swap in general (unless i misunderstood)

I agree. Like @stoppy98 already pointed out:
8Gb RAM is quite sufficient and in my cases, swap is hardly ever used.

Maybe a slight visual warning that RAM is in danger of being full, would suffice and offer to kill some running memory hog; i.e open "top” or one of it's derivatives. :thinking:

2 Likes

I can't say my issues are related to RAM. I'm using different setups from 2GB RAM to 16GB RAM.

But I totally agree, browsers are the worst. Since the whole Desktop experience is moving to the browser (MS O365, Google Tools to name a few), the next step is maybe a 'EliveBrowser'?

The People are able to move from their mobile to their tablet, over the Laptop to the Desktop. Only open a browser and you'll have nearly the same experience, every time ... Welcome to the cloud.

But as far as I know, Elive should support lower systems, that will be thrown away, else. Users (areas?) that are using older systems are not gifted with the best internet. Am I right?
We in the 1st world can choose between an installed Office Suite and the Browser based alternative. I hate to being on tour and can't edit an excel file, because my company is based on any cloud solution and blocked to store files on the host.

I don't think we can avoid the users from want using Facebook, LinkedIn, TickToc, Instagram ... on any machine. Maybe we could help saving RAM by use the user-string and provide the mobile versions?

3 Likes

tha funny fact is that some desktop versions are simpler than the phone one but still using more resources (e.g. Instagram). Though, the mobile applications use A LOT of cached files (between 3 and 6 gb for facebook/instagram). What about, for these having a virtualization support, almost anybody since my 2005 desktop pc supports AMD-V already, running the android apps within some kind of docker? would it be possible? There are already lighter apps alternatives for phones (like facebook light and stuff).

Ah yes and in comparison I remember my Palm Treo phone.
It did everything all current smartphones do but on severely lighter hardware specs (not weight tho :smile:) and very much smaller apps.
It was a joy to see palm-devs code and how they did their best to keep everything to the absolute minimum.
On top battery time was multiple times longer than any current smart phone. :happy_dance:

2 Likes

unlike software now. Lecturers often times even teach "if it works fine and the client doesn't pay you more, tell him to get better hardware. Optimizing is a waste of time". Or "caring about multithreading and multicore is a waste of time unless it is necessarily needed"

1 Like

you may play with these things but i doubt you will have an improvement (which requires a verification from at least 1 week of working to make sure that is really better)

that's what I do with my swap, its on an internal mechanical extra disk (so which is not very used in I/O terms), in order to not waste the few GB of the SSD disk (which could be a faster option for sure)

In a funny way i read on internet a trick to use the GPU (graphic card) memory as a swap system :eyepopping: :rofl2:

yes I think elive needs a feature like this, but no questions about the killing, it should be made automatically :slight_smile: most of the times users has no access / able to do it, so its better to just crash the browser or the graphical server in an extreme situation

:face_vomiting:

welcome to the corrupted capitalist (monetary) world :expressionless: , open-source is a different way to do things but even it, is strongly manipulated by the monetary strings

1 Like

For those who wants to experiment, I found these parameters to work pretty well, minimal lag and minor swap filling up:

vm.swappiness = 20
vm.vfs_cache_pressure = 500

2 Likes

I wasn't aware of that, interesting, I'll do some measures, soon.
I thought the android 'go' apps should address exact this issue on devices with less memory/computing power.
Source: Android (Go edition)

1 Like

I assume that this is possible with the android-x86 OS (is this the correct name? not sure...), which is an android system based on PC architecture, so yes you can run it from a virtualbox, then put the VBOX in hybrid mode and you can run these apps showing directly on your desktop :slight_smile: - the only issue here is that booting an OS also takes a lot of RAM, which can be easily bigger than a browser's tab. Solution?: probably run it from a chroot, like explained here Run applications from any other operating system (in your Elive) , so it doesn't consumes any RAM nor any resouces except the ones of the app itself, I have successfully run songbird from a small debian wheezy chroot :dance:

it's exactly what i was thinking about.

True but often times the bottleneck is the cpu itself when opening boatled pages like youtube and facebook, rather than the ram (i mean anyway when you are browsing facebook or youtube you're not having so many more tabs open!).

It may be an interesting project make a gui manage fot his. really

It actually would be but I can see some caveats using a GUI, like "root" not having access to X11 i.e wrong permissions.

1 Like

maybe we can write a small / amazing howto, "how to run android apps in elive", so:

  • download android-x86
  • uncompress in a chroot
  • run apps from the chroot
  • bingo :partying_face:

I was just thinking on that :smiley:

that should be not hard to resolve :thinking:

noted the idea in my infinite list of tasks to-do

2 Likes

I think i can. Where would you upload the tutorial? Here on the forum?

Yes, as a howto or as a wiki entry where the latter can be edited by others

1 Like

I remember a while back we were talking about how Linux swaps terribly (I think it was @Thanatermesis who called the code garbage).

I found this old message on the BGLUG mailing list and thought might be interesting. NOTE: Observations here were, according to the emailer, were conducted on kernel 4.6 (it's a message from July/August 2019). It may not work on later versions, and I can't really test right now (very very busy) so let me know how it works out for you.

One of the problems I've observed with recent linux kernels is the
difficulty the system has allocating large amounts of memory when it
memory usage is high. The system starts reading swap like crazy, (on
SSD, swap is reading at over 400MB/s), and the system becomes unusable.
In a test on a 8GB system, with firefox running several open tabs, and
then start a 4GB Virtual Machine, swap usage grows to 155MB, but then
the system gets stuck reading from the swap drive at 400MB/s for over 2
hours... (I don't know how long this would have continued, I forced quit
firefox to free ram and liberate the system. Swap never grew past 155MB
during this time.)
This problem only shows up if you're using more memory than you have
RAM, and many people now probably just buy more memory if they encounter
this kind of situation. I, however, do not feel that a system becoming
unusable for going over a few hundred MB makes sense on modern demand
paging system.
At first, I experimented with the /proc/sys/vm/swappiness variable.
Setting this to 100 helped prevent the situation from happening in my
regular workload, (before I introduced a virtualized Windows 10). This
would force the kernel to keep cache size at about 1.5GB, which left
enough available ram to load new allocations used by all my programs.
However, it turns out that the variable I needed to fix was
/proc/sys/vm/watermark_scale_factor
Default value here is 10, it goes up to 1000. After some
experimentation, I figured out that 500 is the value I need to stop my
system form this bad behaviour. Now if I try to allocate a large chunk
of memory, the kernel is able to swap out enough on the fly for the
system to stay functional and stable. (I was actually able to test with
dozens of open firefox youtube instances and a 4GB Virtual Machine.
Total memory usage climbed to well over 11 GB (out of an 8GB system.)
The swapping was crazy, and probably would be death to an SSD to keep
that going forever, but unlike before, the system did not become
unusable while stuck in a perpetual swap reading loop.)

Thanks for the post @TheTechRobo :slight_smile: I'm going to experiment and research about this /proc/sys/vm/watermark_scale_factor option, if it works better, I will set it by default like this in Elive for everybody

Unfortunately my actual computer doesnt' feels much this issue (maybe because my SWAP warning scripts warns me everytime the memory usage grows and I inmediately close the browsers before to get stuck in the hell :rofl2: ), so I invite others that has the same issue of computer blocked to try if that option works better

how to enable it permanently:

sudo su
echo -e "# betatesting https://forum.elivelinux.org/t/computer-block-by-high-ram-usage-swap-saturated/1863/28 :\n# Set the aggressiveness of kswapd to 5%:\nvm.watermark_scale_factor = 500" >> /etc/sysctl.d/swappiness.conf
sysctl --system

IMPORTANT: It is also needed to betatest it with mechanical hard disks, in order to know that is an improvement, the resulting behaviour between an SSD and mechanical disk are giant and entirely different

Interesting read: Tuning the Memory Management Subsystem | System Analysis and Tuning Guide | openSUSE Leap 42.3

note: that original post of @TheTechRobo from a new thread was moved to this one, to still on topic and correct category

1 Like

Once we are in E2*, we can just do the "fake segfault" as described in the thread "how to force shut-down your computer avoiding data loss in Linux" and all their stuff will still be open, I think they might be annoyed if their browser is killed as they might be entering unsaved data :thinking: :thinking: maybe we should not install by default chromium, so that its harder for them to install chromium and slow down their PC than just using the better browser :rofl2: :rofl2:

Yep, everything anyone does is "how can I get money out of this???" they'd do anything for money :expressionless:

It never slows down for me (even at three gigs swap!) but that could be that I use a samsung 860 evo ssd...I didn't test BEFORE the change but judging from your other ssd experiences it has improved :thinking:

Try AltSysrqF if that ever happens again. That calls oom_killer, so it will (?) kill processes using high amounts of RAM (like the browser).

There's also GitHub - rfjakob/earlyoom: earlyoom - Early OOM Daemon for Linux.