View unanswered posts | View active topics It is currently Thu Mar 28, 2024 9:45 pm



Reply to topic  [ 23 posts ]  Go to page 1, 2  Next
 Workstation PI 
Author Message

Joined: Sat Feb 02, 2013 9:40 am
Posts: 2095
Location: Canada
Toying with the idea of using multiple low-cost stock computer boards (sub $100) to build my next workstation from. An issue I see is having OS software running on multiple boards. Is a separate license required for each cpu board? The boards could be networked together, but there needs to be task management software and some means of managing the user interface. Just a single display / keyboard is desired without having to use a keyboard / video switch hardware.

Just a single board could be designated for I/O, but then how to get I/O for tasks running on other boards? For instance suppose I want to run Word word processor on one cpu and Excel spreadsheet on another, while having the desktop managed by yet another cpu board? Could run remote desktop internal to the workstation I suppose.

Contemplating some sort of internal networking like http servers for each board.

_________________
Robert Finch http://www.finitron.ca


Wed Oct 23, 2019 4:33 am
Profile WWW

Joined: Wed Jan 09, 2013 6:54 pm
Posts: 1780
Sounds to me like a cluster - a conventional multi-cpu machine these days has a single shared memory space, whereas a cluster has some interconnected but self-sufficient machines each with cpu and memory. You'd surely need a licensed OS for each such memory space (if it's an open source license, that's still pretty cheap!)

And then, yes, you need to architect some sort of interprocess communication so the tasks can cooperate and access a single shared filing system. Many OSes allow for remote filesystems, so that's not a big issue, but it does mean the one node with the physical filesystem needs to act as a fileserver. Most likely you'd connect your machines with Ethernet and TCP/IP but all sorts of other networks have been used in the past. It's also possible to distribute the filesystem over many nodes, but that's another level of interesting adventure.

Using remote desktop and network shared filesystems is fairly straightforward, I think. It's nice if you can use a protocol whereby an application can put up a window on a remote desktop, instead of having one desktop for each application. X11 can do this, as can Citrix, and I'm sure there are other choices. Not sure if Microsoft's RDP allows for this mode.

I have used compute farms in the past, using Linux and NFS and X11 and a queuing system, but almost all remote jobs were computational jobs, not needing a GUI connection. I think the HPC people do similarly, although they might plumb in a better networking technology and might use a library like MPI to pass messages between jobs.

Ref:
http://talby.rcs.manchester.ac.uk/~ri/_ ... intro.html


Wed Oct 23, 2019 9:51 am
Profile

Joined: Sat Feb 02, 2013 9:40 am
Posts: 2095
Location: Canada
Came across this researching personal cluster computers.
https://hackaday.com/2016/05/09/designi ... l-cluster/
Comments suggest it’s better to use a high-performance single-board multi-core cpu for roughly the same cost.

Seems like it’s going to be a lot of work on the software / configuration side of things, which has me hesitating to try such a thing. I like to use things that are ready right out of the box.
I’ve been looking at boards with a quad-core 1.4 GHz arm processor and 4GB of ram. Linking eight of these together would be 32 cores and 32G’s of ram. More ram and cores than the workstation I have now, and less expensive. A compute cluster would also offer some redundancy. I’m also interested in having the power supply for each board switched off if it’s not in use, so it would use meager amounts of power.

It’s interesting how technologies have moved into the home office with improvements in miniaturization.

_________________
Robert Finch http://www.finitron.ca


Thu Oct 24, 2019 2:45 am
Profile WWW

Joined: Wed Jan 09, 2013 6:54 pm
Posts: 1780
Indeed, things have moved rather a lot. It used to be that a network of transputers with a few k was a worthy target: interprocess communication by message passing. Now you can get 10 to 20 cores in a CPU and 2 to 4 CPUs in a chassis sharing one big memory pool. (I see the record for pi computation used '96 virtual CPUs' which might mean a 48 core box and two threads on each CPU.)

If you think about your 8x4G pi cluster, it's easy to see how you could accommodate as many as 8 processes which each need 3G, but hard to see how you can run even one process which needs 5G of RAM.

AIUI, modern multi-CPU chassis have a high throughput network between the CPU sockets, so that memory requests can be made by one CPU and passed to another one. The RAM is directly connected to one or other CPU, and so a proportion of memory requests are local and a proportion are non-local. This means everything works but memory has a variable cost, and so there's some benefit in carefully placing processes and the corresponding pages onto appropriate CPUs.

Earlier, you'd see a fast wide shared bus, so multiple CPUs made requests to the one bus, and the request went to one or other of several memory cards.

Edit: with reference to the hackaday link, you'll see in that case the end user is running scientific code that they build from source, using appropriate libraries to arrange for message passing between nodes and for synchronisation between cores. That's rather different from someone wanting to run applications provided as binaries which have been built without regard to clusters. In my work history, we mostly ran such closed-source commercial applications, as well some open-source applications, but only in rare cases were the applications written to use multi-core, and never multi-node. As a consequence, we had a range of machine sizes, up to 1TB RAM IIRC, but mostly more like 64G RAM (I forget!) with mostly 2 and sometimes 4 CPUs in a chassis, and mostly dual core in a CPU (it was a while ago.)


Thu Oct 24, 2019 6:37 am
Profile

Joined: Sat Feb 02, 2013 9:40 am
Posts: 2095
Location: Canada
Quote:
If you think about your 8x4G pi cluster, it's easy to see how you could accommodate as many as 8 processes which each need 3G, but hard to see how you can run even one process which needs 5G of RAM.

That would be an issue, especially maybe for games or other software that manages large arrays of data. According to Windows task manager most processes use significantly < 1Gb. It's still tempting to build a low-cost cluster machine for gaining experience with such things. In part there's a hope that future low-cost sbc's would break the 4GB barrier.
Perhaps some sort of virtual machine running on a cluster could be devised such that it looks like it has access to all available memory (cluster machine JVM?). Another possibility is to have an asymmetric cluster with one or two boards with a ton of memory.

An appeal of a cluster computer is expandability, it could be built up over time.

I'd like to get something simple like Tiny Basic running. I had modified Tiny Basic for the 56 node grid computer to have a "run at <node>" command to distribute software but I wasn't able to get it to work. The FPGA based grid computer seemed too unreliable to put more work into it.

_________________
Robert Finch http://www.finitron.ca


Wed Nov 06, 2019 5:11 am
Profile WWW

Joined: Sat Feb 02, 2013 9:40 am
Posts: 2095
Location: Canada
Been thinking about the cluster computer idea again, this time using the Raspberry pi4 as a base. It is a quad-core with 8GB ram available as an option. The larger ram makes the PI more appealing.

So, I go to design my Beowulf cluster computer made up of Raspberry Pi 4s. And I find out that the Raspberry Pi 4 needs a 5V, 3A power supply. It supposedly can use a cell phone charger. But every charger I look at supplies only 5V, 2.4A max and I looked at dozens of chargers. One can buy a charger meant specifically for the PI4 but they’re $6 and its cumbersome because a power-bar is also needed then. I was looking for a charger than supports charging four or more cell phones at once to power four or more PI 4’s.
On the other hand, I see a refurbished desktop computer with 16GB ram is also fairly low cost. The refurbished desktop computer is probably equal in performance to about four Raspberry PI’s and maybe less expensive. Using a second desktop computer for a cluster has a lot of appeal to it.

The issue with my current workstation is that it constantly runs out of memory. It has 16GB ram with a 4GB swap file. The simple solution would be to install more ram.

On the other hand, I desire to build a cluster computer mainly for academic reasons. I am not sure I want to learn the message passing interface and have the whole complexity of the LINUX OS to deal with. I have also thought of building 65C02 computer boards and linking them together with parallel ports. The appeal here being the low power nature of the 65C02. I would rather have a computer that uses mW of power rather than W of power. 5v, 12A (60W+) is a lot for a “toy”.

_________________
Robert Finch http://www.finitron.ca


Sun Jan 10, 2021 11:43 am
Profile WWW

Joined: Wed Jan 09, 2013 6:54 pm
Posts: 1780
Yes, one thing about the various Pi models is that the power supply story isn't as great as one might hope. Your average phone charger may not meet voltage levels when the current demand is taken into account, even supposing it can deliver the current.

All models (other than the Pi 400 keyboard model) can be powered through the 40 pin header, though, and you can use whatever power supply you have which satisfies the voltage and current needs. A cluster computer might well use a PC power supply, with lots of capacity, and that would probably work better and possibly be cheaper than buying a number of USB adapters.


Sun Jan 10, 2021 12:27 pm
Profile

Joined: Sun Dec 20, 2020 1:54 pm
Posts: 74
I did something similar 1 year ago with PPC nodes for a multi service NAS. Not to many nodes, 5 nodes including the main controller node which also acts as "load-balancer". It's not exactly a cluster, and it has a few nodes, but nodes talk over (a simplified) MPI.

robfinch wrote:
the Raspberry Pi 4 needs a 5V, 3A power supply


What about dedicated PSU lanes, one for each node?
You can buy a 60Watt 5V switching PSU, and power feed up to 4 RPI modules.

My "5-PPC-nodes-thing" uses this solution.

robfinch wrote:
it constantly runs out of memory. It has 16GB ram with a 4GB swap file.


I wonder ... what do you have to compute ... for constantly saturating 16GB ram? Here Vivado, Modelsim, a lot of distCC in parallel and even g++ (which is a notorious GB-hungry monster) have never saturated anything but 8Gbyte ram in the worst case.


Sun Jan 10, 2021 12:59 pm
Profile

Joined: Sat Feb 02, 2013 9:40 am
Posts: 2095
Location: Canada
Quote:
I did something similar 1 year ago with PPC nodes for a multi service NAS.
Sounds like an interesting project. PPC nodes are used in super computers.

I did find a 75W 5V PSU. I found a pack of power switching boards using MOSFETs to go along with things. I’ll try and have the root node power on/off other nodes as needed. For now, it would just be experimental, but in the long run I may wish to use a cluster PC as a workstation. If I can get the software to run on it, I would power down parts of the cluster not in use. 8G ram is maybe enough for one copy of Vivado. I can see running an app on each computer. Often times I am just editing files and it seems a waste to have a monster machine running. Why use 80W to edit files when I can use 15W?

I often run more than one copy of Vivado so I can work on one project while another is building. And then I have Word, a text editor, and web browser running too. Then there’s usually multiple copies of Visual studio open at the same time too. In Vivado running the simulator seems to take additional memory. I checked the size of the swap file and Windows had set it automatically to 29GB. So, I manually overrode it to 4GB. Then I reboot when the system runs out memory.

_________________
Robert Finch http://www.finitron.ca


Sun Jan 10, 2021 2:29 pm
Profile WWW

Joined: Sun Dec 20, 2020 1:54 pm
Posts: 74
robfinch wrote:
Sounds like an interesting project. PPC nodes are used in super computers.


As far as I know, Fujitsu A64FX (ARM/64bit based) and IBM POWER9/POWER10 are currently used in supercomputers.

My multi-nodes project uses "embedded processors".

robfinch wrote:
I often run more than one copy of Vivado so I can work on one project while another is building. And then I have Word, a text editor, and web browser running too.


For this stuff I think you don't need a cluster, you need multiple intel-x86 PCs connected to a KVM and using some kind o shared file-system, like NFS, Samba, etc


Sun Jan 10, 2021 4:50 pm
Profile

Joined: Mon Oct 07, 2019 2:41 am
Posts: 585
Must everything be x86? Convert a few PI's just for classic machines.
Ben.


Sun Jan 10, 2021 11:28 pm
Profile

Joined: Sun Dec 20, 2020 1:54 pm
Posts: 74
oldben wrote:
Must everything be x86?
Ben.


You are talking to a person who uses HPPA2 as daily workstation :lol:

That's not the point. If you want to run multiple Vivado sessions, well that's x86 code, so you can't use RPI because Qemu/x86 is not a good deal for performances; I have an x86 accelerator in my RISC workstation, it's 10 times faster than Qemu, but it's much much more expensive than a second hand x86 PC able to comfortably run Vivado.


Mon Jan 11, 2021 1:04 am
Profile

Joined: Sat Feb 02, 2013 9:40 am
Posts: 2095
Location: Canada
Nothing says the cluster must be symmetrical. If I can find a PI-like x86 computer I may eventually add that to the cluster.

I decided to purchase a 5V 20A power supply rather than a cell phone charger, and hack apart some USB cables to power a couple of R PIs. There is some wiring involved anyway as the power is going through MOSFET switches controlled by the root node.
Attachment:
File comment: Cluster Computer
ClusterComputer.png
ClusterComputer.png [ 19.96 KiB | Viewed 1848 times ]

_________________
Robert Finch http://www.finitron.ca


Tue Jan 12, 2021 4:31 am
Profile WWW

Joined: Wed Jan 09, 2013 6:54 pm
Posts: 1780
Be aware that the Pi likes to see 5V at the supply, so use thick enough wiring to avoid too much voltage drop. (If the FETs drop a voltage, look after that too!)


Tue Jan 12, 2021 8:55 am
Profile

Joined: Sun Dec 20, 2020 1:54 pm
Posts: 74
That's not a "cluster", it's some "computers in a rack", so find/build/buy a mini rack cabinet, with PC racks, a KVM and a power distributor.


Tue Jan 12, 2021 12:34 pm
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 23 posts ]  Go to page 1, 2  Next

Who is online

Users browsing this forum: Applebot, Bing [Bot] and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group
Designed by ST Software