| Pages: 1 2 3 4 5 6 7 8 9 10 [11] 12 13 14 :: one page |
|
|
| Author |
Topic |

Fabrezio
Deep Core Mining Inc.
 |
Posted - 2008.07.07 17:27:00 -
[301]
Originally by: CCP Lingorm OK some general points again.
Multiple Cores/multi threading can not solve all the problems.
Do you really want a multi-threaded combat engine? Your using a more complex setup than me and your hit/damage calculations take longer to perform, than mine on a simpler setup, but because of multi-threading mine are threaded out and complete first so my damage is applied to you first, even though when you damage calculations are completed it turns out I am dead .... not really the situation we want.
There are some things we can do about this using wait locks so that we gather up all the damage/hit calculations to be processed in a time slice, the process then waits for all the responses to come back and then continues to do the linear processing of all the results, so we spread out the processing load to multi-thread the bits that can be parallel processed. But this would have to be tested to the nth degree to make sure that the timeslice size is correct and multithreading and wait process works. And to be honest I would not like to start this with the combat engine .... something a bit less complex would be nicer and safer. Say the physics engine.
This is something we are looking into, but as you can understand it is not as easy as chucking in a couple of thread calls and recompiling.
I am hoping to have some nice news on this front soon (or getting the actual dev's involved in this work to write a blog).
A couple of things: 1) We are talking about fractions of a second faster, not minutes, hours or days. As long as both are applied it doesn't matter the order. Furthermore, the limiting factor would be the cycle of the weapon and not the complexity of the calculation (as one is 4 orders of magnitude longer).
2) This demonstrates a general ignorance about multi-threading and parallelizing in general, which is disappointing.
3) You can break the problem down and apply something similar to tomasulo's algorithm:
Algorithm
More importantly, the concept of a re-order buffer:
ROB
What does this all mean? You transformed the problem (there really isn't a problem, but if they want strict control over when damage is applied, this would do it) from something that requires locks up the wahzoo into a very simple producer / consumer model, which is well understood. Please do not remove moderator edits from your signature. It has been removed because it doesn't reflect your ingame persona. If you have comments or questions please email us at mods@ccpgames.com. Thanks, Hango |

Fabrezio
Deep Core Mining Inc.
 |
Posted - 2008.07.07 17:37:00 -
[302]
I would like to add one more thing:
Please goto one of your local universities and hire a consultant, someone with a PhD. This would solve many of your problems faster than having your engineers search in the dark like blind moles. Please do not remove moderator edits from your signature. It has been removed because it doesn't reflect your ingame persona. If you have comments or questions please email us at mods@ccpgames.com. Thanks, Hango |

Sazumaan Johnza
Minmatar Southern Cross Incorporated Southern Cross Alliance
 |
Posted - 2008.07.14 14:34:00 -
[303]
Originally by: Fabrezio
...hire a consultant, someone with a PhD. This would solve many of your problems faster than having your engineers search in the dark like blind moles.
Erm........are you quite sure about that? 
How long do you think such a consultant would have to "search in the dark" to understand the issue? Do you seriously think there are consulting firms with such specific expertise?
Besides, the hourly rates paid over a couple of months would mean CCP wouldn't be able to afford to upgrade *anything*...!  "Eve is more filling than roast steaks slowly grilling over a rotating fire whilst marinating in a combination of Australian fruity wines and the best imported herbs..." - SChimera [16.4.07] |
|

CCP Lingorm
C C P

 |
Posted - 2008.07.14 15:06:00 -
[304]
Originally by: Fabrezio
A couple of things: 1) We are talking about fractions of a second faster, not minutes, hours or days. As long as both are applied it doesn't matter the order. Furthermore, the limiting factor would be the cycle of the weapon and not the complexity of the calculation (as one is 4 orders of magnitude longer).
2) This demonstrates a general ignorance about multi-threading and parallelizing in general, which is disappointing.
3) You can break the problem down and apply something similar to tomasulo's algorithm:
Algorithm
More importantly, the concept of a re-order buffer:
ROB
What does this all mean? You transformed the problem (there really isn't a problem, but if they want strict control over when damage is applied, this would do it) from something that requires locks up the wahzoo into a very simple producer / consumer model, which is well understood.
As I have pointed out, these are some of the general problems that we face with multihreading. I am not doing the actual implementation so do not know the details of the actual work that is being done. I know the general details and that is what I have passed on to the people here because they were interested.
So I am going to have to say, that the stuff you point out is very cool and I will have a look in my spare time, for personal interest, but I can not comment on the implementation.
My knowledge on this project is from the testing point of veiw in that designing a test framework that will allow us to prove that the changes are better/strong/faster and that certain things (like killing ships after you have died, except with missile and all the other variables) can be tested in a reproducible way. So I have to understand the concepts if not the fine details (though I will most liekly pick that up as we test different stuff).
How this helps.
CCP Lingorm CCP Quality Assurance QA Engineering Team Leader
Originally by: Lord Fitz Eve is to WoW as Wow is to an 8 player game of Unreal Tournament.
|
|

Jim McGregor
 |
Posted - 2008.07.14 15:25:00 -
[305]
Edited by: Jim McGregor on 14/07/2008 15:25:28
Nice thread, very interesting stuff. Should be stickied in the Eve technology forum or something.
Also thanks a lot to the OP for all the nice links to the dev posts!
---
Originally by: Roguehalo Can you nano Titans?
|

RedClaws
Amarr Dragon's Rage Intrepid Crossing
 |
Posted - 2008.07.14 15:37:00 -
[306]
Edited by: RedClaws on 14/07/2008 15:37:56 Would CCP happen to be looking at Venus Blue software to help them?
|

Beltantis Torrence
 |
Posted - 2008.07.14 18:42:00 -
[307]
Edited by: Beltantis Torrence on 14/07/2008 18:46:10
Originally by: Nova Fox Why not C#?
For absolute speed C/C++ wins. C# is better than Python however, but I doubt the backend is written in Python (or at least I hope it isn't). Compiler optimizations are huge when it comes to sheer computational power. You want tail end recursion unraveled and inlined and as far as I know Python just isn't capable of doing that in the binary sense due to its interpreted model.
I think part of the issue with players making recommendations is that we don't know enough about the backend systems to make recommendations.
For instance, in terms of multithreaded order of execution issues mentioned earlier in this thread (time slices/sync objects verse FIFO), I have to question why a lot of these calculations are being done at all. So I mean my first instinct would be to take a look at the setup and see how much memory I had to work with (or RAM disks or what have you for ultra-fast caching) and then figure out at what point I could take computations entirely out of the equation with a large indexed pre-calculated table. Since Eve uses % and high precision for a lot of its calculations this becomes a battle of how much accuracy you'd lose by pre-calcing assuming you couldn't store every possible permutation. If that accuracy would fall under the realm of noticable in gameplay then you'd remove the computation altogether and you'd have fast IO responsible instead. Ideally you'd want to put all load on an equal distribution basis and do FIFO so you could eliminate as many sync objects as possible (as they act more or less like traffic signals for the data).
However, I couldn't make that recommendation short of throwing blind guess without understanding better what the load issue is. Might be nice if there was some sort of mechanism that'd allow us to understand and make intelligent suggestions without being burdensome for the dev team at CCP.
Edited to add shameless plug - I'm a partner in a consulting firm which, amongst other things, ports systems into cloud computing scenarios and does performance optimizations. Just incase CCP is interested in outsourcing some of the analytical pieces ;).
|

Beltantis Torrence
 |
Posted - 2008.07.14 20:06:00 -
[308]
Edited by: Beltantis Torrence on 14/07/2008 20:14:22
Originally by: Semkhet If you allocate resources per system, you cannot preempt overload conditions since you act AFTER a player appears in a context. If you allocate resources first to each player and then modulate these resources according to the context and actual load (space or docked, and amount of people in system), you can design a naturally adaptive load balancing mechanism. You can preempt load conditions since the server knows where a given player wants to jump BEFORE executing said jump.
...
Hence there should be a master clock linked to a variable amount of temporary slave clocks created within these interactive contexts (let's say one slave clock per grid), and these slave clocks should be allowed to slightly alter their frequency according to load conditions in order to evenly spread latency among ALL actors.
Its surprisingly hard to get synchronization down to that fine of a grain. The system/grid route is the right route to take, just needs to be made faster. The only way what you're mentioning would work in the context of a game would be if you could avoid any synchronization whatsoever (IE, real time processing).
|

Lowa
Gallente North Star Networks Black Hand.
 |
Posted - 2008.07.21 16:00:00 -
[309]
Edited by: Lowa on 21/07/2008 16:01:10
This is a nice thread, there has been many like them but this one is good. And make me happy in pants. :)
Question: What is the top 3 most "resource" taking elements in a fight? - Loading grid? - 4000 drones buzzing about? - Damage calculations? - Profit?
As I did see some discussion on damage I go out on a limb and guess that doing damage calulations is a pretty tough job for the CPU/software? In regards to that, and sorry if it has been asked/answered before; - Would it help if damage was done at "set levels" based on skill, distance, weapon/ammo type etc? Example; - 425mm Rail Gun II - Level 5 on all relevant gunnery skills - Level 5 on ship skill - No damage mods
Damage @ 0km to 50km range = 225HP using Antimatter Damage @ 150km to 200km range = 115HP using Spike
- 10 more damage / damage mod
At a certain distance using certain skills, guns and ammo you would do 225HP damage. Not 223.3 or 34.1 or... Then no calculations would need to be done that are based on speed, angle, fall-off etc.
PLEASE NOTE THAT I SEE MANY MANY MANY FLAWS IN MY OWN ARGUMENTS HERE! 
I think what I'm trying to say is; can it exist a way to reduce the number of calculations needed in battle without screwing up years of shield/armor resistance training? (not to mention the Nano lovers)  And even IF that was the case, would we as the players accept that for working, large scale battles?
Again, thanks for this thread, keep it going!
/Lowa
|

Kaeten
Hybrid Syndicate
 |
Posted - 2008.07.22 00:52:00 -
[310]
ccp need more hamsters... yea they definitely need more hamsters.. ________________________ I'M POOR
|
|

Xaziar Nortocus
Forces Unknown Inc
 |
Posted - 2008.07.22 20:36:00 -
[311]
Originally by: Kaeten ccp need more hamsters... yea they definitely need more hamsters..
Nah, just a viable means of reducing internal and external lag...Infiniband can do it....but with massive recoding and infrastructure changes....I don't expect it for at least a year... --------------------------------------------------
 |

Thargat
Caldari North Star Networks Black Hand.
 |
Posted - 2008.07.22 22:12:00 -
[312]
Originally by: Lowa Edited by: Lowa on 21/07/2008 16:01:10
This is a nice thread, there has been many like them but this one is good. And make me happy in pants. :)
Question: What is the top 3 most "resource" taking elements in a fight? - Loading grid? - 4000 drones buzzing about? - Damage calculations? - Profit?
As I did see some discussion on damage I go out on a limb and guess that doing damage calulations is a pretty tough job for the CPU/software? In regards to that, and sorry if it has been asked/answered before; - Would it help if damage was done at "set levels" based on skill, distance, weapon/ammo type etc? Example; - 425mm Rail Gun II - Level 5 on all relevant gunnery skills - Level 5 on ship skill - No damage mods
Damage @ 0km to 50km range = 225HP using Antimatter Damage @ 150km to 200km range = 115HP using Spike
- 10 more damage / damage mod
At a certain distance using certain skills, guns and ammo you would do 225HP damage. Not 223.3 or 34.1 or... Then no calculations would need to be done that are based on speed, angle, fall-off etc.
PLEASE NOTE THAT I SEE MANY MANY MANY FLAWS IN MY OWN ARGUMENTS HERE! 
I think what I'm trying to say is; can it exist a way to reduce the number of calculations needed in battle without screwing up years of shield/armor resistance training? (not to mention the Nano lovers)  And even IF that was the case, would we as the players accept that for working, large scale battles?
Again, thanks for this thread, keep it going!
/Lowa
Stop stalking me in interesting threads and come join the party (you know of what I speak of). 
There's only one sig that matters... and that's Radius. |

Napro
Caldari Paxton Federation
 |
Posted - 2008.07.27 11:18:00 -
[313]
Originally by: Beltantis Torrence Edited by: Beltantis Torrence on 14/07/2008 18:46:10
Originally by: Nova Fox Why not C#?
For absolute speed C/C++ wins. C# is better than Python however, but I doubt the backend is written in Python (or at least I hope it isn't). Compiler optimizations are huge when it comes to sheer computational power. You want tail end recursion unraveled and inlined and as far as I know Python just isn't capable of doing that in the binary sense due to its interpreted model.
I think part of the issue with players making recommendations is that we don't know enough about the backend systems to make recommendations.
For instance, in terms of multithreaded order of execution issues mentioned earlier in this thread (time slices/sync objects verse FIFO), I have to question why a lot of these calculations are being done at all. So I mean my first instinct would be to take a look at the setup and see how much memory I had to work with (or RAM disks or what have you for ultra-fast caching) and then figure out at what point I could take computations entirely out of the equation with a large indexed pre-calculated table. Since Eve uses % and high precision for a lot of its calculations this becomes a battle of how much accuracy you'd lose by pre-calcing assuming you couldn't store every possible permutation. If that accuracy would fall under the realm of noticable in gameplay then you'd remove the computation altogether and you'd have fast IO responsible instead. Ideally you'd want to put all load on an equal distribution basis and do FIFO so you could eliminate as many sync objects as possible (as they act more or less like traffic signals for the data).
However, I couldn't make that recommendation short of throwing blind guess without understanding better what the load issue is. Might be nice if there was some sort of mechanism that'd allow us to understand and make intelligent suggestions without being burdensome for the dev team at CCP.
Edited to add shameless plug - I'm a partner in a consulting firm which, amongst other things, ports systems into cloud computing scenarios and does performance optimizations. Just incase CCP is interested in outsourcing some of the analytical pieces ;).
CCP Needs to hire this guy
|

bloomich
Caldari In Siders
 |
Posted - 2008.07.27 11:26:00 -
[314]
Hi,
Just a firendly question. Not a technical one on purpose. All I was going to ask was...
Roughly, how long do you estimate it might be before a solution to reducing or eliminating fleet lag is implemented?
I understand predicting the future is impossible, sometimes what might be 6 months away might actually be 2 years away. However, I am thinking that fleet lag might be sorted in around 2-3 years time.
|

Napro
Caldari Paxton Federation
 |
Posted - 2008.07.27 11:39:00 -
[315]
Edited by: Napro on 27/07/2008 11:43:51
Originally by: bloomich Hi,
Just a firendly question. Not a technical one on purpose. All I was going to ask was...
Roughly, how long do you estimate it might be before a solution to reducing or eliminating fleet lag is implemented?
I understand predicting the future is impossible, sometimes what might be 6 months away might actually be 2 years away. However, I am thinking that fleet lag might be sorted in around 2-3 years time.
Don't hold your breath. Really seems like the only way is to completely rewrite everything rather than throw more hardware at it. And every time a CCP dev speaks up, it's usually to say "it's too hard"
So I really doubt they'll ever put the effort necessary into it. We're going to have to wait til they get some legit competition to push them into fixing it or maybe even the competition will get it right and they'll have a nice exodus on their hands.
Maybe Jumpgate Evolution will be the swift kick in the ass CCP needs
|

Tamoko
Damage Unlimited Inc INTERDICTION
 |
Posted - 2008.08.11 22:16:00 -
[316]
Edited by: Tamoko on 11/08/2008 22:18:30
Originally by: Napro
Originally by: Beltantis Torrence Edited by: Beltantis Torrence on 14/07/2008 18:46:10
Originally by: Nova Fox Why not C#?
For absolute speed C/C++ wins. C# is better than Python however, but I doubt the backend is written in Python (or at least I hope it isn't). Compiler optimizations are huge when it comes to sheer computational power. You want tail end recursion unraveled and inlined and as far as I know Python just isn't capable of doing that in the binary sense due to its interpreted model.
I think part of the issue with players making recommendations is that we don't know enough about the backend systems to make recommendations.
For instance, in terms of multithreaded order of execution issues mentioned earlier in this thread (time slices/sync objects verse FIFO), I have to question why a lot of these calculations are being done at all. So I mean my first instinct would be to take a look at the setup and see how much memory I had to work with (or RAM disks or what have you for ultra-fast caching) and then figure out at what point I could take computations entirely out of the equation with a large indexed pre-calculated table. Since Eve uses % and high precision for a lot of its calculations this becomes a battle of how much accuracy you'd lose by pre-calcing assuming you couldn't store every possible permutation. If that accuracy would fall under the realm of noticable in gameplay then you'd remove the computation altogether and you'd have fast IO responsible instead. Ideally you'd want to put all load on an equal distribution basis and do FIFO so you could eliminate as many sync objects as possible (as they act more or less like traffic signals for the data).
However, I couldn't make that recommendation short of throwing blind guess without understanding better what the load issue is. Might be nice if there was some sort of mechanism that'd allow us to understand and make intelligent suggestions without being burdensome for the dev team at CCP.
Edited to add shameless plug - I'm a partner in a consulting firm which, amongst other things, ports systems into cloud computing scenarios and does performance optimizations. Just incase CCP is interested in outsourcing some of the analytical pieces ;).
CCP Needs to hire this guy
Not to be overbearing or rude, but everything he said is basically Computer Science 101. I was taught it all over a decade ago in Grade 10. It's all still applicable today.
Originally by: Lowa Question: What is the top 3 most "resource" taking elements in a fight? - Loading grid? - 4000 drones buzzing about? - Damage calculations? - Profit?
As I did see some discussion on damage I go out on a limb and guess that doing damage calulations is a pretty tough job for the CPU/software? In regards to that, and sorry if it has been asked/answered before; - Would it help if damage was done at "set levels" based on skill, distance, weapon/ammo type etc?
I can't rightly say how damage calculations compare to loading a grid or all those little buzzing drones. But floating point (layman's terms: math that requires real numbers) arithmatic is definitely costly, so with over ten-thousand sources of damage creating regular damage "events" (say, 400 vs 400, 8 guns, 5 drones = 10'400 sources of damage, worst case scenario) that's got to add up awfully quick.
@ Lowa: factors like skill all collate into the modules "damage multiplier" which is constant. That's not being calculated for every damage event that gets thrown. Likewise resists, et alii are (read: should be) locally available to the server. It shouldn't have to do any legwork to find all that stuff out, or calculate it on the fly.

Originally by: "Bellum Eternus" Is 'speed tank' code for 'completely invulnerable'?
|

Locke DieDrake
Human Information Virus
 |
Posted - 2008.08.11 23:04:00 -
[317]
I don't want to rain on the parade... but Floating point is NOT any kind of barrier to fixing lag. If there is one thing modern servers are good at, it's FLOPS. Lots and lots of FLOPS.
Damage calcs may play a role in the lag issue, but it isn't for lack of available horsepower.
My desktop computer gets around 2500 FLOPS on a bad day. I could optimize it for a lot more.
I don't know the exact specs on the EVE blades, but I'm guessing they are pushing 5k FLOPS each. Possibly a lot more if they use a hardware sub system to do FLOP (think CUDA for a modern idea of this, or a math co-processor for an older version).
If EVE is lagging because of Floating Point math, there is a SERIOUS bug in the coding somewhere. My guess, it's more to do with basic bandwidth (not internet connection, data bandwidth on the server bus). Just because you can calculate 10k floating point operations per second, doesn't mean you can send that data where it needs to go.
______________________________________________ Goon FC(08/12/06):"its a trap" "that thing is fully operational" |

Tamoko
Damage Unlimited Inc INTERDICTION
 |
Posted - 2008.08.11 23:54:00 -
[318]
I wont pretend to know what I'm talking about with blade servers, but the problem appears to be the architecture of the software: It's not utilizing the blades hardly as well as it could/should/would if...
It's easy to say that their computers are really, really fast. Obviously, they're fast enough to do floating point math. But something is bogging right down during 800 man fleet battles. Even if calculating ten thousand sources of damage isn't proportionately large, having a means to spin that algorithm off to a separate code path with threading/infiniband/whatever can only help.

Originally by: "Bellum Eternus" Is 'speed tank' code for 'completely invulnerable'?
|

Janey Talor
 |
Posted - 2008.08.12 00:44:00 -
[319]
Originally by: Locke DieDrake
I don't know the exact specs on the EVE blades, but I'm guessing they are pushing 5k FLOPS each.
If they are the same blade servers they bought about a year and a half ago, I think they were Intel P4 Xeons, so 1000-2000k MFlops.
Anyhow, summarising the suggestions so far
- Buy bigger / more servers - Offload floating point to Cuda (GPU) - Lookup tables for floating point calculations - Thread the code and benefit from Dual / Quad etc core - Thread the code and use MPI to split the processing across nodes (insert terms like, HPC, Grid Computing, Infiniband et al) - Go and write a new MMORPG about Zombies instead screw you guys 
|

Reptzo
Channel 4 News Team
 |
Posted - 2008.09.01 18:36:00 -
[320]
Any chance we can get an update? And a real dev blog about it? |
|

Eri Chell
 |
Posted - 2008.09.15 10:17:00 -
[321]
So basically (only read like the first page) all this is, is new "bridges" between the differnt nodes in the cluster? Or is this a "bridge" between the CPU and HDs? Or both?
|

Heather Drauls
Rabies Inc.
 |
Posted - 2008.09.16 14:14:00 -
[322]
Originally by: Eri Chell So basically (only read like the first page) all this is, is new "bridges" between the differnt nodes in the cluster? Or is this a "bridge" between the CPU and HDs? Or both?
You can think of Infiniband as a system bus that reaches to other systems. So it 'bridges' CPU/Disk/Memory on ALL machines that are connected to the same IB fabric, kinda giving you one single machine if you've written your code to take advantage of it.
 |

Ishina Fel
Caldari Synergy. Imperial Republic Of the North
 |
Posted - 2008.09.16 20:14:00 -
[323]
Hi CCP Lingorm,
I've been wondering, has the topic of stream processing been brought up internally in regards to the troubles with porting the combat engine to modern multi-core CPUs?
Stream processing, so writes wikipedia, is a form of parallel processing that eschews "allocation, synchronization, or communication" between the processing units.
In other words, the problem with multiple CPU cores interfering with each other does not exist in this programming paradigm - or at least I think this is the point of the whole thing. Please be aware that my understanding of this is very limited, and this whole post is based on that possibly flawed understanding. 
Anyway: while this may sound outlandish to some, stream processing is actually becoming quite popular nowadays, and relevant hardware is easily available to everybody: modern video cards are nothing but highly powerful stream processing units, with each shading unit being one processing core - for example, ATI's latest chips have 800 of them, each! Makes a quad-core CPU look rather humble, no...? Another example is the Cell CPU used in the Playstation 3 console.
Now, the fun thing is: EVE's server code is largely written in C, if I got that right. And NVidia's GPGPU language model, CUDA, is nothing but a modified C compiler! I'm sure it would take recoding, but much much less than other suggestions made in this thread. CCP probably wouldn't even have to invest serious money into retraining their programmers (or hire new ones).
Plus, stream processing is fast. In fact, it is mind-boggingly fast. For this purpose, I'd like to showcase the stats page of Folding@home, a distributed computing project that uses people's home computers to fold proteins for research purposes. They were among the first to jump on the GPGPU train, and have clients that can make use of both NVidia and ATI video cards for the calculations.
The page dynamically upodates, but at the time of this writing, 4,100 ATI chips generate 450 teraflops computing power, and 12,700 Nvidia chips generate another 1,400 teraflops. There are Playstation 3 clients too; and even though their Cell processors are weaker than the graphics chips, the 40,000 of them manage another 1,150 teraflops.
And then there's our generic CPUs. Around 220,000 cores, and among them are state-of-the-art quadcores and server racks full of opterons. Together, they manage 210 teraflops.
Again: 17,000 video cards generate 1,850 teraflops. 220,000 x86-CPUs generate 210 teraflops.
That is the power of stream processing. And it carries through to other applications; video converters using CUDA have reported over 400% performance gains over traditional programs, and so on. Google a bit around for applications of GPGPU, and the trend is clear.
The question remains: Can Tranquility run on stream processors? Can the EVE server code be modified to take advantage of this particular method of computing, or is it incompatible? If it could be done, the issue of coding for multiple CPU cores may become a thing of the past, because strwam processing doesn't care how many units it has at its disposal, and which unit does what. Plus, video cards can be effortlessly linked in pairs, triplets or quads, with the cards themselves managing the load distribution. And even limited to one card, the sheer parallel processing power is so monstrous compared even to a quad-core (let alone a single core) that bottlenecks will be a thing of the past.
Can it be done, Lingorm? Has it been brought up internally? Did I even understand the whole concept right? Note: even the #1 supercomputer in the world, Roadrunner, has a stream processing cluster...
 |

Reptzo
Channel 4 News Team
 |
Posted - 2008.09.16 20:54:00 -
[324]
Ishina, you bring up an excellent point with stream processing, but it has some problems. I have rather limited knowledge in this area and won't pretend I am well versed. That being said, I know that many things do not port well to video card stream processing. I know that folding@home does not port well, even tho its gets high results. It takes the video cards 4 computations to get the same result as 1 CPU computation, due to the inefficiencies of the GPU. That being said it shows the raw power of the video cards that even having to do the same problem 4 times to get the same results as a regular CPU, it is still leaps and bounds better.
So, while the processing may be faster, there is another problem. Nvidia and ATI are still battling for who will be the preferred GPGPU platform, along with Intel making an appearance in 2 to 3 years with their own CPU/GPU chip. While these systems may create huge performance gains, going to them may prove fatal in the long long. Until these systems are more widely accepted and established, one of the three companies could change their efforts, cancel their stream processing lines, or any number of things that happens to bleeding edge technology. If CCP chose the wrong company, it may put them in quite the pinch.
All that being said, I am in no position to say whether stream processing is even feasible or desirable for the eve server. SO, comment from CCP Linghorn would be good.
|

Kazuo Ishiguro
House of Marbles Zzz
 |
Posted - 2008.09.16 22:12:00 -
[325]
Originally by: Ishina Fel Stream processing, so writes wikipedia, is a form of parallel processing that eschews "allocation, synchronization, or communication" between the processing units.
Groups of solar systems are run on nodes (i.e. individual CPU cores). Each of these nodes has to be in constant communication with lots of other nodes, as people move around from one to another; it's not like rendering an image, a task which can be split into independent pieces that can all be worked on simultaneously. Also, we're not yet at the stage, afaik, where a single stream processor is more capable than the best available CPU core.
Regarding the code I suspect that there are significant sections of non-C code that would not be so easy to replace. ----- DIY copying in Liekuri 20:1 mineral compression Nomad JF for sale |

Ishina Fel
Caldari Synergy. Imperial Republic Of the North
 |
Posted - 2008.09.17 09:31:00 -
[326]
Originally by: Reptzo That being said, I know that many things do not port well to video card stream processing.
Yes, there's that. It's basically why I'm asking if the topic has been raised internally at CCP; if anyone knows if their code is viable for porting, then it would be them! And indeed, GPGPU is very much a brute force method at the moment... it lacks the 30 years of constant development and refining that x86 CPUs have undergone already. But just think of the potential that such development may still unlock in stream processors!
Originally by: Reptzo Nvidia and ATI are still battling for who will be the preferred GPGPU platform, along with Intel making an appearance in 2 to 3 years with their own CPU/GPU chip. While these systems may create huge performance gains, going to them may prove fatal in the long long. Until these systems are more widely accepted and established, one of the three companies could change their efforts, cancel their stream processing lines, or any number of things that happens to bleeding edge technology.
Actually, Intel's Larrabee is not a stream processor, but rather a many-core x86 CPU. It brings with it all the issues and increased effort of writing code for multi-core CPUs. Granted, Intel may provide graphics drivers which take over the load balancing between the cores, but that'll be graphics only... plus, the individual cores are actually pretty weak. The Larrabee has much less potential in GPGPU than existing video chips.
The third company that offers stream processors is IBM with their PowerXCell line of CPUs, as used in PS3's and the Roadrunner cluster I linked. I'd find it hard to believe that a company which is already supplying large-scale supercomputers would suddenly decide to not pursue that approach any further. In fact, the Roadrunner cluster is a perfect example of how it could work: it has a series of x86 CPUs for normal work, but also a series of stream processing units that specifically get assigned the very math intensive calculations that would take far too long on a x86 core. Tranquility could also be a hybrid machine, with dedicated stream processing units just for managing fleet fights - IF, as mentioned, the code can even work in that way.
Originally by: Kazuo Ishiguro Groups of solar systems are run on nodes (i.e. individual CPU cores). Each of these nodes has to be in constant communication with lots of other nodes, as people move around from one to another; it's not like rendering an image, a task which can be split into independent pieces that can all be worked on simultaneously. Also, we're not yet at the stage, afaik, where a single stream processor is more capable than the best available CPU core.
Actually, they /are/ more capable than a x86 core... in specific task sets. As mentioned above, not everything is fit to be ported to stream processing, and a single stream unit may not have the sophistication of 30-years-refined x86 technology... but there's a ton of them in any single core, and they have a brute force power that leaves a x86-integrated FPU flopping and gasping like a fish on dry land.
Plus, the day-to-day operation of the EVE cluster doesn't have to be ported. It's just the few cases where a CPU gets bogged down with intense math, as it happens in fleet fights. If the combat engine alone could be ported, and a couple dedicated stream processing racks connected via InfiniBand would be waiting to handle all combat offloaded to them, a large part of the code and TQ cluster wouldn't even need to be changed...
...but only CCP knows if it's possible. 
 |

Kazuo Ishiguro
House of Marbles Zzz
 |
Posted - 2008.09.17 09:52:00 -
[327]
I'm not sure whether the current bottleneck is in the process for generating the sets of combat calculations, or the calculations themselves. If it's the latter, I suppose it might be possible to farm them out to stream processors, and then back again, once infiniband is in place. i.e.
A tries to fire on B ... Node receives command CPU sends calculation request to stream proc Result comes back CPU propagates result to A & B
Of course, it might be that except in very big battles, this is a lot slower than doing everything on the CPU alone, because of the two extra transmissions. ----- DIY copying in Liekuri 20:1 mineral compression Nomad JF for sale |

Pan Crastus
Anti-Metagaming League
 |
Posted - 2008.09.17 10:42:00 -
[328]
Originally by: Kazuo Ishiguro I'm not sure whether the current bottleneck is in the process for generating the sets of combat calculations, or the calculations themselves. If it's the latter, I suppose it might be possible to farm them out to stream processors, and then back again, once infiniband is in place. i.e.
A tries to fire on B ... Node receives command CPU sends calculation request to stream proc Result comes back CPU propagates result to A & B
Of course, it might be that except in very big battles, this is a lot slower than doing everything on the CPU alone, because of the two extra transmissions.
Stream processors and stream processing are worthless for EVE. They are designed to process large amounts of data and perform identical operations on that data in parallel. A good workload is usually graphics/audio/physics stuff.
See further up for the "Actor Model", which is a much better way to handle games with many objects / MMOs in general.
How to PVP: 1. buy ISK with GTCs, 2. fit cloak, learn aggro mechanics, 3. buy second account for metagaming
|
|

CCP Lingorm
C C P

 |
Posted - 2008.09.17 11:26:00 -
[329]
The EVE Servers are Blades, which means that they do not have GPU's or the ability to fit GPU's into them.
GPGPU based processing does offer some advantages, but to make use of it we would have to radically increase the size and complexity of our server environment over what it currently is.
We still watch the advances in this technology, but have no current plans to make use of it for EVE.
If things change and it becomes viable then we would relook at it as an option.
CCP Lingorm CCP Quality Assurance QA Engineering Team Leader
Originally by: Lord Fitz Eve is to WoW as Wow is to an 8 player game of Unreal Tournament.
|
|

Ishina Fel
Caldari Synergy. Imperial Republic Of the North
 |
Posted - 2008.09.17 12:00:00 -
[330]
IBM offers Cell CPU blades 
But I get what you mean: CCP has considered it, and decided it isn't practicable at the moment. Too bad.
Still, thanks for you continued responses tothis thread, it makes it more awesome than the last dozen devblogs combined! 
 |
|
| Pages: 1 2 3 4 5 6 7 8 9 10 [11] 12 13 14 :: one page |
| First page | Previous page | Next page | Last page |