Pages: 1 2 3 4 5 6 :: [one page] |
|
Author |
Thread Statistics | Show CCP posts - 16 post(s) |
|
CCP Zymurgist
Gallente C C P
|
Posted - 2010.12.11 15:19:00 -
[1]
CCP Masterplan is revealing part two of his dev blog. This time he shares some of the technical details and the results of the improvements here.
Zymurgist Community Representative CCP Hf, EVE Online Contact Us |
|
Mashie Saldana
Minmatar Veto Corp
|
Posted - 2010.12.11 15:23:00 -
[2]
Sweet, more blog ****.
|
Tharill daSai
Caldari Serringer Arms Inc Free United Spirits
|
Posted - 2010.12.11 15:29:00 -
[3]
Blogs good
|
Trebor Daehdoow
|
Posted - 2010.12.11 15:29:00 -
[4]
Well done, Team Gridlock!
Confessions of a Noob Starship Politician The most expensive free trip to Iceland you'll ever win!
|
TeaDaze
|
Posted - 2010.12.11 15:35:00 -
[5]
Masterplan, best plan
TeaDaze.net Blog | CSM Database |
Sgt Blade
Caldari Save Yourself Inc. Imajiaca
|
Posted - 2010.12.11 15:37:00 -
[6]
how interesting
Hypnotic Pelvic Thrusting Level 5 |
Yon89
Northern Coalition.
|
Posted - 2010.12.11 15:38:00 -
[7]
Thanks now our drake fleets of d00m will rock even more ============= SIG SIG SIG |
Saana Nara
|
Posted - 2010.12.11 15:38:00 -
[8]
Good blog, I'd love to get more of this behind the scenes info on development :)
|
Masjheira
Minmatar Cursed Inc. Mercenary Coalition
|
Posted - 2010.12.11 15:39:00 -
[9]
Can you please remove Drakes from the game? This way, no more missile blobs and no more CPU issues ! :P
Thank you very much.
|
Phoenix Tyrox
Krupp-Stahl Majesta Empire
|
Posted - 2010.12.11 15:41:00 -
[10]
Thanks for this awesome DevBlog! It covers what I missed in the last DevBlogs: Behind-the-scene insides in form of pseudo code. I really enjoyed this one and I am looking forward to seeing more of this kind.
|
|
Alexeph Stoekai
Stoekai Corp
|
Posted - 2010.12.11 15:43:00 -
[11]
Originally by: TeaDaze Masterplan, best plan
qft -----
|
Julian Assagne
|
Posted - 2010.12.11 15:50:00 -
[12]
Dev Blog with Graphs = Win.
-Julian Assagne CEO of EVELeaks
|
Orkanen
|
Posted - 2010.12.11 15:55:00 -
[13]
This was great. It's always nice to get some insight into why the game behaves the way it does. Also, programmer war stories are good stuff.
|
Skuggis
Systembolaget
|
Posted - 2010.12.11 15:56:00 -
[14]
Good job! I find information like this to be super interesting, keep it coming.
|
DmitryEKT
Point of No Return Waterboard
|
Posted - 2010.12.11 15:56:00 -
[15]
Glad you managed to fix this. Now fix the drake. If it wasn't so OP compared to other BCs for these blobfests it wouldn't have been an issue in the first place.
|
static zero
Minmatar Power of the Phoenix
|
Posted - 2010.12.11 15:57:00 -
[16]
How time-consuming are the operations to acquire data about balls in the field? Could some of the data be acquired asynchronously?
Great stuff, thanks for sharing :D
-static
-static zero |
Marchocias
Snatch Victory
|
Posted - 2010.12.11 15:57:00 -
[17]
Who's the man?
Masterplan! ---- I belong to Silent Ninja (Hopefully that should cover it). |
Salpun
Gallente Paramount Commerce
|
Posted - 2010.12.11 16:01:00 -
[18]
The new fight on lag page places the smooth fleet fight limit at 600. What it does not say is if that is on a reinforced blade or not. Could you tell us? And the limit on the other blade type? Also with these new changes can you push that higher or do you have to have an actual fight on TQ that is with in standerd ie no lag to justify moving the number to the right.
|
Dunpeal
Caldari Caldari Provisions
|
Posted - 2010.12.11 16:05:00 -
[19]
Edited by: Dunpeal on 11/12/2010 16:05:39
Originally by: DmitryEKT Glad you managed to fix this. Now fix the drake. If it wasn't so OP compared to other BCs for these blobfests it wouldn't have been an issue in the first place.
Actually it is my opinion that it is quite the opposite, without the obvious fame of the one ship (regardless of wich), this particular event that resulted in this fix, would've lacked that catalyst to put all into motion, wich stands to reason to question if the bottleneck would've been so easy as identified or even called to atention to search for that one needle in the humungous haystack that is eve's code.
So overall eve is now better, BECAUSE of the supposed overpoweredness of that one ship, so i gues... be thankfull for it. -------------------------------------------------
My Past, my destiny... |
Jason Edwards
Internet Tough Guy Spreadsheets Online
|
Posted - 2010.12.11 16:09:00 -
[20]
Just be honest... the server code is all in pseudo code and thats why its so bad? ------------------------ To make a megathron from scratch, you must first invent the eve universe.
|
|
Motriek
|
Posted - 2010.12.11 16:12:00 -
[21]
Since a copy of the serialized data is now being sent to all clients, do we (clients/observers) still see personalized transversal figures on our overview? Transversal (radians/sec) differs for every observer.
|
Gnulpie
Minmatar Miner Tech
|
Posted - 2010.12.11 16:12:00 -
[22]
Wooooooot!
Awesome stuff.
And presented in an even better way. That is just great.
*applauds* Well done Team Gridlock and CCP Masterplan!
(The original loop was a bit silly though ) |
ElfeGER
Deep Core Mining Inc.
|
Posted - 2010.12.11 16:15:00 -
[23]
nice improvement :)
is that SinglecastByClientID full CPU usage or more network delay while pushing the updates to the proxies?
are the GUI guys also using the profiler now? (the overview window in large fleets needs some love)
|
Kesi Raae
|
Posted - 2010.12.11 16:16:00 -
[24]
Does this mean we can bring twice as many Drakes to fights, now?
|
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.11 16:19:00 -
[25]
Originally by: static zero How time-consuming are the operations to acquire data about balls in the field? Could some of the data be acquired asynchronously?
Individually, querying this data isn't too expensive. It does add up though if we need to query the data lots of times in the same frame. That's why I added the single-frame cache: If we need to look up some ball info for multiple observers, the first one will cause the value to be fully looked-up/calculated, and subsequent requests will pull the results from the cache. As I mentioned, this was more of a problem when one fleet warps into another. In these cases, I was seeing a cache hit:miss ratio of 50,000:1 (I'm talking about the ball-state cache here, not the CPU cache) This is less of a problem (and so less of a saving) in non-warp situations.
Originally by: Salpun The new fight on lag page places the smooth fleet fight limit at 600. What it does not say is if that is on a reinforced blade or not. Could you tell us? And the limit on the other blade type? Also with these new changes can you push that higher or do you have to have an actual fight on TQ that is with in standerd ie no lag to justify moving the number to the right.
Those numbers are based on a reinforced node. On a regular node, if all the other systems are empty, it will be almost the same as a reinforced node. It is impossible to say what a regular node will support in normal operation, as it all depends on what is happening in the other systems it is currently hosting.
At the moment that marker based on load graphs, player feedback, simulations and some trend-analysis. Quantifying 'gameplay quality' is somewhat impossible. However we're currently developing some more tangible metrics, such as 'How many modules are cycling on a node' and 'Module activation delay'. Depending on how reliably these correlate with actual gameplay experience, we might even publish these for everyone to see at some point. However this is all rather experimental at this point, so I'm not promising anything -- Team Gridlock: Herding electronic cats since 2010 |
|
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.11 16:26:00 -
[26]
Originally by: ElfeGER nice improvement :)
is that SinglecastByClientID full CPU usage or more network delay while pushing the updates to the proxies?
are the GUI guys also using the profiler now? (the overview window in large fleets needs some love)
Thanks! SinglecastByClientID is still full CPU burn doing serialisation/addressing right now. Reducing this is somewhere on our list (roughly sorted by gain/effort) The actual IO interaction is handled asynchronously by stacklessIO. Telemetry was initially integrated into the client for just this sort of thing. When Gridlock saw a preview of it, it was very much a case of "WANT ON SERVERS NOW PLEASE!" -- Team Gridlock: Herding electronic cats since 2010 |
|
DeftCrow Redriver
Gallente Best Path Inc.
|
Posted - 2010.12.11 16:30:00 -
[27]
Originally by: TeaDaze Masterplan, best plan
Woohoo!
|
Ebisu Kami
|
Posted - 2010.12.11 16:34:00 -
[28]
Originally by: CCP Masterplan [...]
Did you guys notice that, according to the graphs, the "Missiles - Inventory and Damage"-part of the load increased by roughly 12% from old to new?
|
Gant Grief
|
Posted - 2010.12.11 16:35:00 -
[29]
Hehe, pseudocode
|
Perkuno Sunus
Gallente Unicorn Enterprise Rooks and Kings
|
Posted - 2010.12.11 16:35:00 -
[30]
Edited by: Perkuno Sunus on 11/12/2010 16:34:58 Both of your articles were very interesting/awesome! They gave a bit of insight how things work internally. It's always interesting to see, how things were, and what kind of optimizations can be applied or trade-offs chosen from in a specific area to achieve the desired result. And of course, most importantly - the observable improvement. I hope this is not the last blog of this kind. Thank you.
|
|
Turix
Interstellar eXodus BricK sQuAD.
|
Posted - 2010.12.11 16:41:00 -
[31]
I must say, this is exactly the kind of information I love to read about.
Personally, being a software engineer myself, I do like the detail (Psuedo-Code + Graphs = Win!). Defiantly keep writing these types of blogs and if possible encourage other teams to do the same, with the same level of detail! It would be... awesome? __________________________
|
Salpun
Gallente Paramount Commerce
|
Posted - 2010.12.11 16:42:00 -
[32]
Thanks for the reply. Great blog.
To some reinforced means that you are using the jita super node. Is this the case? If not what performance are you expecting out of it the next time you use it on a major fleet fight? Also how many of those do you have now and when was the last time they where used out side of jita?
|
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.11 16:47:00 -
[33]
Originally by: Ebisu Kami Did you guys notice that, according to the graphs, the "Missiles - Inventory and Damage"-part of the load increased by roughly 12% from old to new?
I was waiting for someone to spot that! During the first test, the node was running close to 100%. Therefore the launcher modules weren't necessarily cycling at their full rate for every shot (the drakes were on the edge of experiencing module lag) Once the CPU was lowered, the launcher were able to cycle at their full rate all the time, and so were able to pump out more missiles during the 100 seconds. More missiles means more inventory operations, so absolute inventory load increases slightly. CCP Veritas' earlier blog explores some of these artifacts that occur at the edge of performance limits (and beyond) -- Team Gridlock: Herding electronic cats since 2010 |
|
Arkady Sadik
Minmatar Electus Matari
|
Posted - 2010.12.11 16:47:00 -
[34]
I would like to thank CCP and the devs for writing these very technical devblogs. I'm a programmer myself and I really appreciate such a close look at the work of the programmers at CCP. It really gives a "oh my gods, I can so totally relate to that" feeling. Thank you! :-)
|
Ebisu Kami
|
Posted - 2010.12.11 16:49:00 -
[35]
Originally by: CCP Masterplan
Originally by: Ebisu Kami Did you guys notice that, according to the graphs, the "Missiles - Inventory and Damage"-part of the load increased by roughly 12% from old to new?
I was waiting for someone to spot that! During the first test, the node was running close to 100%. Therefore the launcher modules weren't necessarily cycling at their full rate for every shot (the drakes were on the edge of experiencing module lag) Once the CPU was lowered, the launcher were able to cycle at their full rate all the time, and so were able to pump out more missiles during the 100 seconds. More missiles means more inventory operations, so absolute inventory load increases slightly. CCP Veritas' earlier blog explores some of these artifacts that occur at the edge of performance limits (and beyond)
I see. Thanks for the reply and clarifying :)
|
Zaotome
|
Posted - 2010.12.11 16:51:00 -
[36]
thank you! i really like detailed info + pics :)
|
Gripen
Rage and Terror Against ALL Authorities
|
Posted - 2010.12.11 16:59:00 -
[37]
What is the point of sending new update for every missile launched? Why can't you do it in the same manner as with turrets: send only "activate\deactivate launcher" updates with RoF of launcher specified so both client side and server side simulations could create new missile objects for themselves at specified interval.
|
|
CCP Explorer
|
Posted - 2010.12.11 17:02:00 -
[38]
Originally by: Salpun Thanks for the reply. Great blog.
To some reinforced means that you are using the jita super node. Is this the case?
Reinforced simply means that the solarsystem was placed alone on a dedicated node. Each blade in the TQ cluster has a 4 cores (and 32 GB RAM) and we therefore run 4 nodes on each blade.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
Salpun
Gallente Paramount Commerce
|
Posted - 2010.12.11 17:06:00 -
[39]
Thanks for the reply but you dodged the hard part of the question.
|
Amy Garzan
Gallente The Warp Rats
|
Posted - 2010.12.11 17:57:00 -
[40]
Awesome to see the results of this work knowing it has been going on for quite a while and in multiple parts, and also to see how its linking together.
So when do we get to see pictures of the server move a few months back as promised?? -------------------------------------------------- 101010 The Answer to Life, The Universe, and Everything |
|
static zero
Minmatar Power of the Phoenix
|
Posted - 2010.12.11 18:13:00 -
[41]
Originally by: CCP Masterplan That's why I added the single-frame cache: If we need to look up some ball info for multiple observers, the first one will cause the value to be fully looked-up/calculated, and subsequent requests will pull the results from the cache. As I mentioned, this was more of a problem when one fleet warps into another. In these cases, I was seeing a cache hit:miss ratio of 50,000:1 (I'm talking about the ball-state cache here, not the CPU cache)
Oh! I glossed over that three-line paragraph. That's a striking hit ratio, no pun intended. So the single-frame cache helps populate the ball-field list. Then that is used build an update for each observer of the ball-field.
Later, you mention that serialization of that data needs to be done only once, and can be done using that list. On your bar graph, that appeared to be the bigger savings (unless SendToClient implies some other work to be done).
It sounds to me like each observer no longer has ball-field adds/deletes built just for them. Instead, they are now "subscribed" to ball-field updates (with adjustments for corner cases). Makes a lot of sense.
All of this sounds like a multicast paradigm to me, but that's probably only because I'm a network guy. :)
Well done!
-static -static zero |
Tester128
|
Posted - 2010.12.11 18:16:00 -
[42]
nice blog, man good work with optimization too keep it up!
|
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.11 18:19:00 -
[43]
Edited by: Daneel Trevize on 11/12/2010 18:20:50 Going to repeat myself from part 1, but, re: Destiny being 1Hz
Can you ever lock something that mwd+cloaks within the same second/tick that you ctrl+click to lock them with, say, a preactivated point/other mods? |
An ewb
|
Posted - 2010.12.11 18:27:00 -
[44]
If everything is happening with 1hz, but everyone in the node is experiencing increasing delay, wouldn't it be possible to slow down the simulation itself dynamically based on how much stress the node is under? Have a fluff piece like "Massive fleet formation in this area is causing temporal disturbances" and slow down the simulation cycle so everyone still gets fed the stuff that is happening, but everything just seems to be going slower if there's more stuff happening, instead of random dropoffs and blackscreenings and such. Is this just a dumb idea?
|
The Mach
Reikoku IT Alliance
|
Posted - 2010.12.11 18:33:00 -
[45]
Great Blog!! Can nVidia Physx be used to optimize this further, or at least shift the load to a currently unused graphics card?
|
Dr Laren
|
Posted - 2010.12.11 18:39:00 -
[46]
Originally by: The Mach Great Blog!! Can nVidia Physx be used to optimize this further, or at least shift the load to a currently unused graphics card?
Servers don't have graphic cards.
|
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.11 18:40:00 -
[47]
Edited by: Daneel Trevize on 11/12/2010 18:40:18 They already blogged that the issue's pre-tick, not the crunching of any physics simulation calculations but just setup and preparing to disseminate the results of said calcs. Also, Physx can go DIAF. Open GPGPU less so. |
Dunpeal
Caldari Caldari Provisions
|
Posted - 2010.12.11 18:51:00 -
[48]
Edited by: Dunpeal on 11/12/2010 18:52:07
Originally by: An ewb If everything is happening with 1hz, but everyone in the node is experiencing increasing delay, wouldn't it be possible to slow down the simulation itself dynamically based on how much stress the node is under? Have a fluff piece like "Massive fleet formation in this area is causing temporal disturbances" and slow down the simulation cycle so everyone still gets fed the stuff that is happening, but everything just seems to be going slower if there's more stuff happening, instead of random dropoffs and blackscreenings and such. Is this just a dumb idea?
No, it isn't, not because i say so, but because mastermind said so on the comment thread of the first part. It has gotten lots of looking into that as a "solution" to lag, not to kill it but to allow the node to "breath". Mainly why they havent already jumped head first into it seems that it is more complicated than it seems to implement, when a solar system starts to get purpusefully out of sync with the rest of the cluster alot of planing needs to be done to allow for that temporal distortion to happen and make sure the node plays nice with all the other nodes. -------------------------------------------------
My Past, my destiny... |
Nareg Maxence
Gallente
|
Posted - 2010.12.11 18:56:00 -
[49]
I love your stuff!
Using a profiler can be a huge eye opener. I have done some profiling on a DB application. By figuring out exactly where the bottlenecks where I managed to get 80-90% speed increases on some routines.
You are definitely up there competing with PrismX as my favorite dev!
|
Carniflex
StarHunt R.A.G.E
|
Posted - 2010.12.11 19:15:00 -
[50]
I think I have noticed in the past that 1 Hz tick of the nodes by experimenting with damage modules on ships. So a little clarification would make me a bit happier
Say, I have weapon system that with 3 damage mods has RoF of 4.2 seconds and with 4 damage mods RoF of 3.95 seconds. If I can kill the target in one volley does this mean that in the first case I can reactivate my gun on next target at 5th second and on second case of 4th second or can I activate my weapon system mid tick as well at 4.2 seconds ? Or is RoF of 4.2 seconds only used when it takes several volleys to kill a target and in one volley situations the RoF is rounded up to nearest full second ?
|
|
Black Dranzer
Caldari
|
Posted - 2010.12.11 19:27:00 -
[51]
Edited by: Black Dranzer on 11/12/2010 19:29:26
Originally by: Dev Blog 1: for every observer: # (ie a user's ship) 2: for every ball in the observer's bubble: 3: if ball was added/deleted to the bubble in the last tick: 4: gather up all the data required to tell the observer's client about this ball
Originally by: Dev Blog step 4 always produces the same data for a given ball, regardless of who the observer is
Originally by: CCP, paraphrased Let's perform redundant calculations inside a quadratic loop OH GOD WHY IS OUR CODE SO SLOW
|24 Hour Plex|Mining Makeover| |
Louis deGuerre
Gallente Amicus Morte Dead Muppets
|
Posted - 2010.12.11 19:33:00 -
[52]
Tech blogs make me feel all warm and fuzzy ----- Amicus Morte is recruiting. Dive into the world of 0.0 !
|
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.11 19:33:00 -
[53]
Edited by: Daneel Trevize on 11/12/2010 19:34:34
Originally by: Black Dranzer WHY WOULD YOU DO THAT?
Because premature optimization is evil, so first you get it to work, and then you have other things to do |
Black Dranzer
Caldari
|
Posted - 2010.12.11 19:39:00 -
[54]
Originally by: Daneel Trevize Edited by: Daneel Trevize on 11/12/2010 19:34:34
Originally by: Black Dranzer WHY WOULD YOU DO THAT?
Because premature optimization is evil
This particular situation does not support that theory.
|24 Hour Plex|Mining Makeover| |
Chruker
|
Posted - 2010.12.11 20:09:00 -
[55]
Not to rain on your parade, but I've been getting a laggy experience in ex. Agrallarier and other systems surrounding Auvergne. It seems to have been in the last couple of weeks and it has been with just around a 100 people in the systems.
The lag is especially noticable when salvaging and looting missions... it is not severe lag, but more of a 1-2 seconds delay on every action. ----- http://games.chruker.dk/eve_online ----- Top wishes: - No daily downtime - Faster training on sisi
|
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.11 20:40:00 -
[56]
Edited by: Daneel Trevize on 11/12/2010 20:41:47
Originally by: Black Dranzer This particular situation does not support that theory.
It just worries me; For the longest time I assumed that the kind of stuff that was causing lag was just general scale errors that would only be solvable by extremely clever algorithms. But if that kind of stuff has been sitting in the code base for however many years? God only knows what else is in there that needs tidying up. I mean, look, I do not claim to be an elite programmer, but when I'm going through a large collection and calculating stuff based off of that collection, assuming I'm actually writing field code and not just some function test prototype stuff, the first thing I do is look for major redundancies.
But, hey, nobody's paying me however many dollars an hour to program, so there we go.
Sadly I learnt that in RL with budgeted time & money if you dare show most employers that something works at all, they'll have the idea it's done and there's far higher priorities, and I expect it goes doubly when developing a game who's progress is what the company depends on to keep bringing in the iskies, rather than in say, an institution that's 'too big to fail' and you would think could afford to adopt a policy of doing it right first time...
Incidentally my fave Java look & feel style is the one that makes everything look like building blueprints rather than a regular GUI. |
Charles Javeroux
Gallente INTERSTELLAR CREDIT
|
Posted - 2010.12.11 21:40:00 -
[57]
This blog gave me inspiration...."Save New Eden, kill a missile ship TODAY!"
Originally by: Orek Fear I guess the ultimate solution to inflation in EVE turned out to be an NPC stripper...
Originally by: Concerned Capsuleers Why can't we just be left alone?!
|
Hylax Ciai
Cataclysm Enterprises Ev0ke
|
Posted - 2010.12.11 22:07:00 -
[58]
Edited by: Hylax Ciai on 11/12/2010 22:07:52 Seeing that you put a nested loop in there i just couldn't think of anything else than "Man. Today is so loopy." Unfortunately i've seen just a few episodes of the series..
|
Ben Derindar
Dirty Deeds Corp.
|
Posted - 2010.12.11 22:53:00 -
[59]
Excellent read, nice work.
|
EdwardNardella
Capital Construction Research
|
Posted - 2010.12.11 22:55:00 -
[60]
This would be a worthy second half of the Christmas gift mentioned in the blog about learning skills. CCRES is recruiting pilots who want to live in WSpace/Wormholes. Fill out an application here! |
|
Mirei Jun
Einherjar Rising Cry Havoc.
|
Posted - 2010.12.11 22:55:00 -
[61]
Good work, with good explanation.
This is what I would expect from a "Dev Blog", and hope its used more exclusively for this kind of writing in the future.
|
Chruker
|
Posted - 2010.12.11 23:00:00 -
[62]
Originally by: Mirei Jun Good work, with good explanation.
This is what I would expect from a "Dev Blog", and hope its used more exclusively for this kind of writing in the future.
Unfortunately then the dumb people will come whining about the devblogs being too technical ----- http://games.chruker.dk/eve_online ----- Top wishes: - No daily downtime - Faster training on sisi
|
Dwindlehop
Stimulus Rote Kapelle
|
Posted - 2010.12.11 23:00:00 -
[63]
This is an excellent blog. Thanks!
|
Kei Nagase
Minmatar Applied Creations The Fendahlian Collective
|
Posted - 2010.12.12 00:12:00 -
[64]
I enjoyed this type of Balg Sigerceptors Skill Bonus: 5% reduction in Sig Radius per level |
Bob Niac
Gallente Fink Operations -Mostly Harmless-
|
Posted - 2010.12.12 00:20:00 -
[65]
Moar?
|
Siena Petrucis
Caldari German Kings Majesta Empire
|
Posted - 2010.12.12 00:31:00 -
[66]
Usally, pilots in a big fleet fight have all weapon effects turned off. So that means there are close to zero observers for missiles. It would be a waste to send all the updates about created and destroyes missiles to clients which do not render it. Does the server take that into account? If not, would it be a possible way for more optimization?
|
Khenar
|
Posted - 2010.12.12 00:45:00 -
[67]
Edited by: Khenar on 12/12/2010 00:45:59 May I ask how the single-frame cache works with respect to cloaked ships? It's been mentioned before during a Fanfest that when an observer in a grid receives a client update, that update has been stripped of all information regarding cloaked ships within that grid except the observer's own ship, so that the observer could not just then analyze the update and figure out where the other cloaked ships are.
During the previous system, this seems easy to handle given that the serialization operation happened individually for each observer. But now it seems that this operation only happens once per frame. How do you distinguish the updates for those cloaked observers? From the blog, it looks like you don't store the cloaked ship operations in that cache: "Most clients in the same bubble will received a serialised update containing the same information. Why serialize that information for every single client, when I have already determined the part they all have in common?"
This leads me to think that you are still performing individual serialization operations for each cloaked observer so that their client updates include the update information of their cloaked ship. How does this affect the performance of the node if a significant number of cloaked observers were introduced into a grid?
Edit -- Minor grammar error.
|
Zenon Mu
Advanced Assemblies and Sciences
|
Posted - 2010.12.12 00:52:00 -
[68]
awesome! I love reading about something getting fixed :) |
Infinion
Caldari Endless Destruction Imperial 0rder
|
Posted - 2010.12.12 00:52:00 -
[69]
Originally by: Salpun Thanks for the reply but you dodged the hard part of the question.
Originally by: Salpun Thanks for the reply. Great blog.
To some reinforced means that you are using the jita super node. Is this the case? ?
The jita super node is for Jita, if ccp removed it to reinforce a fleet fight for 24 hours between a few alliances imagine what kind of whiplash we would see on the markets
|
Karbon Dating
|
Posted - 2010.12.12 00:55:00 -
[70]
OMG. This had to be one of the best blogs ever. As a CS major, info like this really hits the spot. I know exactly what you are talking about.
I want MORE!
|
|
Zendoren
Aktaeon Industries
|
Posted - 2010.12.12 01:00:00 -
[71]
Originally by: CCP Masterplan
Originally by: Ebisu Kami Did you guys notice that, according to the graphs, the "Missiles - Inventory and Damage"-part of the load increased by roughly 12% from old to new?
I was waiting for someone to spot that! During the first test, the node was running close to 100%. Therefore the launcher modules weren't necessarily cycling at their full rate for every shot (the drakes were on the edge of experiencing module lag) Once the CPU was lowered, the launcher were able to cycle at their full rate all the time, and so were able to pump out more missiles during the 100 seconds. More missiles means more inventory operations, so absolute inventory load increases slightly. CCP Veritas' earlier blog explores some of these artifacts that occur at the edge of performance limits (and beyond)
With the code not currently taking advantage of multi-threading and/or multi-core processing, I'm assuming that when you reinforce the soler system (kick the other 3 soler system off the node) that you are doing it more for the memory than the processing power?
|
galphi
Gallente Furnulum pani nolo
|
Posted - 2010.12.12 01:36:00 -
[72]
You could also replace the physical object missiles with the 'fake' missile effects used by the fighter-bombers. It's not like anyone ever uses defender missiles anyway
|
FzZZy
Gallente Biceps Superstars
|
Posted - 2010.12.12 02:28:00 -
[73]
Thanks for a good devblog. I realy like when you at CCP share your small eureka moments. You mention that the increase in inventory and explosions was due to not maxing the preformence. Is it the same case why evolve increased with 1/2 (>10ms)?
The serialization of a ball that happen every tick, can't it be cached to the next tick whare the ones that is updated is reserialized? Or does the a ball update to often?
|
Chssmius
Helljumpers En Garde
|
Posted - 2010.12.12 02:57:00 -
[74]
Loved the blog and would appreciate more of this type.
Originally by: Black Dranzer Edited by: Black Dranzer on 11/12/2010 19:29:26
Originally by: Dev Blog 1: for every observer: # (ie a user's ship) 2: for every ball in the observer's bubble: 3: if ball was added/deleted to the bubble in the last tick: 4: gather up all the data required to tell the observer's client about this ball
Originally by: Dev Blog step 4 always produces the same data for a given ball, regardless of who the observer is
Originally by: CCP, paraphrased Let's perform redundant calculations inside a quadratic loop OH GOD WHY IS OUR CODE SO SLOW
[whywouldyoudothatpicture]
I have to agree. Aside from appearing to be more straight forward to type and think about I can't think of a reason to write the code that way(he says without looking at the real code).
Originally by: galphi You could also replace the physical object missiles with the 'fake' missile effects used by the fighter-bombers. It's not like anyone ever uses defender missiles anyway
As I understand it there are a problems with that. Chiefly, the fact that the fighter-bombers are now basically using guns instead of missiles.
Take The EvE Personality Test today! |
Neo160
|
Posted - 2010.12.12 04:06:00 -
[75]
question about missiles. Pre-Tyrannis (in Dominion i think), if a missile was flying slow enough, and far enough away, i could click the missile and actually track it with my camera (pretty cool), this ability was removed after Tyrannis released. was this a lag fighting decision, and if so, whats the CPU impact from being able to track missiles with the in-game camera? just curious
great blog, very informative, as I'm in school for this kinda stuff, it helps to be able to try to wrap my head around programing code and logic without worrying about deadlines or grades , DeVry goes through stuff really fast, and they only touch on the basics, if at all.
|
Galius Zed
|
Posted - 2010.12.12 04:23:00 -
[76]
Double-cheers and yay for pseudo code!
vOv
|
Aquana Abyss
|
Posted - 2010.12.12 05:31:00 -
[77]
Wouldn't a better solution just be to remove the drake from game?
In fact, remove missiles too.
|
Rok Qhang'Rawl
Joint Espionage and Defence Industries
|
Posted - 2010.12.12 06:22:00 -
[78]
Good Fight!
The interesting work of optimization is more about figuring out where to make changes first than the details of the solution, but you made both parts rich and meaningful here. Nice work. Thanks for sharing.
|
|
CCP Explorer
|
Posted - 2010.12.12 09:13:00 -
[79]
Originally by: Jason Edwards Just be honest... the server code is all in pseudo code and thats why its so bad?
Pseudo code is the only thing the hamsters understand.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.12 09:15:00 -
[80]
Originally by: Motriek Since a copy of the serialized data is now being sent to all clients, do we (clients/observers) still see personalized transversal figures on our overview? Transversal (radians/sec) differs for every observer.
They are calculated by the client based on the movement vectors.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
|
CCP Explorer
|
Posted - 2010.12.12 09:23:00 -
[81]
Originally by: Daneel Trevize
Originally by: Black Dranzer WHY WOULD YOU DO THAT?
Because premature optimization is evil, so first you get it to work, and then you have other things to do
Indeed. First you write readable code that works. Then, based on measurements, you optimise the parts that need to be optimised.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.12 09:29:00 -
[82]
Originally by: Siena Petrucis Usally, pilots in a big fleet fight have all weapon effects turned off. So that means there are close to zero observers for missiles. It would be a waste to send all the updates about created and destroyes missiles to clients which do not render it. Does the server take that into account? If not, would it be a possible way for more optimization?
All the clients in the bubble need to have the state of the simulation because they process it in sync with the server. There are also other reasons for the client to know about missiles than to render them, such as the interaction of missiles with AoE weapons.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.12 09:35:00 -
[83]
Originally by: galphi You could also replace the physical object missiles with the 'fake' missile effects used by the fighter-bombers. It's not like anyone ever uses defender missiles anyway
You have to remember that missiles interact with AoE weapons. We have given this significant thought but it's not trivial to implement.
You will note that even if the fighter bomber graphical effect is missile-like and even if the fighter bomber damage is calculated missile-style then you can't use AoE weapons to defend against their missiles.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
Lan Staz
|
Posted - 2010.12.12 10:14:00 -
[84]
Originally by: Zendoren With the code not currently taking advantage of multi-threading and/or multi-core processing, I'm assuming that when you reinforce the soler system (kick the other 3 soler system off the node) that you are doing it more for the memory than the processing power?
I believe (hope a dev will correct me if I'm wrong here) that the "reinforced" status is applied to the node, not the blade; one of the 4 nodes on one of the blades now handles just one solar system.
In the general, non-reinforced, case you have many solar systems per node, and 4 nodes per blade (one for each CPU).
|
Elsa Nietzsche
|
Posted - 2010.12.12 12:10:00 -
[85]
one of if not the sexiest blog ever. thanks for this great one. keep up the great work!
|
Gnulpie
Minmatar Miner Tech
|
Posted - 2010.12.12 12:27:00 -
[86]
Edited by: Gnulpie on 12/12/2010 12:35:43 Good speedup, this is pretty impressive.
But why does the machoNet::SinglecastByClientID take that long? The Destiny::SendToClients without the SingleCast is only 7.4 ms! So the overhead is not in sorting out the data any more but in sending it. Is it possible to delegate this to a (highly) parallel worker thread which is doing all the work in the background (on other nodes maybe?) or do you get in trouble then with the non-blocking paradigm of Destiny?
Edit: Do you lookup always SessionID's from ClientID's in each SingleCast? That would be a huge waste in this case as they are almost always constant. |
Typhado3
Minmatar
|
Posted - 2010.12.12 13:08:00 -
[87]
very good to hear about this stuff being done. Also nice to see explanations and graphs.... however that first graph in part 2 the label on the y axis is seconds?? thought it would be cpu ------------------------------ God is an afk cloaker |
Karak Terrel
As Far As The eYe can see Chained Reactions
|
Posted - 2010.12.12 13:52:00 -
[88]
Saw the algorithm and was like "Oh you can just read it. They must use ruby now" then i realized that is was pseudocode to make language X readable. -- please consider to visit our w-space system, cake will be served immediately. |
Altaree
The Graduates Morsus Mihi
|
Posted - 2010.12.12 14:08:00 -
[89]
This type of dev blog is one of the reasons I joined and still play eve. As a developer myself I love to read this sort of thing! --Altaree
|
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.12 14:15:00 -
[90]
Time for another wall-of-text response session :)
Originally by: Daneel Trevize Edited by: Daneel Trevize on 11/12/2010 18:20:50 Going to repeat myself from part 1, but, re: Destiny being 1Hz
Can you ever lock something that mwd+cloaks within the same second/tick that you ctrl+click to lock them with, say, a preactivated point/other mods?
That probably just depends on how the dice roll regarding network packet ordering and tasklet scheduling. Chances are: probably not, but I can't say for certain.
Originally by: Carniflex Say, I have weapon system that with 3 damage mods has RoF of 4.2 seconds and with 4 damage mods RoF of 3.95 seconds...
Module activations/deactivations are handled by a different system (Dogma). Dogma doesn't have a discrete tick-rate like Destiny, it simply runs as often as it can and processes as many effects as it can in a reasonable amount of time. Dogma effects might have consequences, such as adding a missile into space. During its next tick Destiny will then pick up that a missile has been added, and perform the necessary additions/deletions step to inform the clients. Even though missile can only be added to space on a 1Hz tick, the actual module cycling will still correctly use sub-second precision. (Until things get laggy, and then cycling might get delayed)
Quote: On why wasn't this done in the first place?
This is a two-part answer: Firstly, as a start-up with minimal budget, simply getting to launch is priority number one. Sure, you could spend a lot of time projecting where your hotspots might be if you are very successful, but by then you will have ran out of time and money, and have no releasable product.
Secondly, when it is laid out in several lines of psuedocode, it becomes very easy to see what is wrong. However those few lines represent many hundres of lines of real code, spread across multiple modules, that have evolved over time to do many things. It was quite a lot of work to understand the low-level elements, and then to see how they piece together, finally forming the full picture. It is entirely possible that initially it was written in a different way altogether. Piece-by-piece it was changed over time, eventually arriving at where it was before I began looking into it.
Originally by: Hylax Ciai Edited by: Hylax Ciai on 11/12/2010 22:33:37 So, the main reason for missiles being items and not just effects, is that their damage is not applied instantly?
Also, seeing that you put a nested loop in there i just couldn't think of anything else than "Man. Today is so loopy." Unfortunately i've seen just a few episodes of the series..
Correct. Time-of-flight is what makes missiles an interestingly different mechanic compared to turrets. Also, I see what you did there. Nice find -- Team Gridlock: Herding electronic cats since 2010 |
|
|
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.12 14:18:00 -
[91]
Edited by: CCP Masterplan on 12/12/2010 14:25:04
Originally by: Khenar This leads me to think that you are still performing individual serialization operations for each cloaked observer so that their client updates include the update information of their cloaked ship. How does this affect the performance of the node if a significant number of cloaked observers were introduced into a grid?
You are correct, there are a number of edge-cases that I simplified out of the explanation for the sake of space. There is still a small part of each update that can't be pre-serialised (which is why SendToClients doesn't collapse down to a very small time). Cloaking and a few others details are handled here.
Originally by: FzZZy Thanks for a good devblog. I realy like when you at CCP share your small eureka moments. You mention that the increase in inventory and explosions was due to not maxing the preformence. Is it the same case why evolve increased with 1/2 (>10ms)?
The serialization of a ball that happen every tick, can't it be cached to the next tick whare the ones that is updated is reserialized? Or does the a ball update to often?
Evolve actually varies a fair bit between frames (In that particular test, it was in the range 25-40 ms if I recall) That is why I recorded the first graph over 100 seconds, to smooth out this variation. As for re-using the serialisation across ticks - it is impossible to determine if something else modified some ball state outside of the Destiny loop. I suppose we could do something with dirty-flags, but I'm not sure the saving would be too much.
Originally by: Neo160 question about missiles. Pre-Tyrannis (in Dominion i think), if a missile was flying slow enough, and far enough away, i could click the missile and actually track it with my camera (pretty cool), this ability was removed after Tyrannis released. was this a lag fighting decision, and if so, whats the CPU impact from being able to track missiles with the in-game camera? just curious
great blog, very informative, as I'm in school for this kinda stuff, it helps to be able to try to wrap my head around programing code and logic without worrying about deadlines or grades , DeVry goes through stuff really fast, and they only touch on the basics, if at all.
I'm not sure about the 'why' of that change - I wasn't involved in it. However camera tracking is purely client-side, so it wouldn't have changed anything as far as the server is concerned.
Originally by: Typhado3 very good to hear about this stuff being done. Also nice to see explanations and graphs.... however that first graph in part 2 the label on the y axis is seconds?? thought it would be cpu
It is actually seconds spent inside timer areas, which I grouped into the six high-level categories. However you could also consider it as CPU allocation if you like - in this case they are pretty much equivalent. -- Team Gridlock: Herding electronic cats since 2010 |
|
zcar300
Gallente
|
Posted - 2010.12.12 17:31:00 -
[92]
Interesting. Keep up the good work.
|
MotherMoon
Huang Yinglong
|
Posted - 2010.12.12 19:06:00 -
[93]
Edited by: MotherMoon on 12/12/2010 19:15:17
Originally by: CCP Explorer
Originally by: Siena Petrucis Usally, pilots in a big fleet fight have all weapon effects turned off. So that means there are close to zero observers for missiles. It would be a waste to send all the updates about created and destroyes missiles to clients which do not render it. Does the server take that into account? If not, would it be a possible way for more optimization?
All the clients in the bubble need to have the state of the simulation because they process it in sync with the server. There are also other reasons for the client to know about missiles than to render them, such as the interaction of missiles with AoE weapons.
ok lets be honest, It doesn't feel like an intended feature to have smartbombs randomly take down missiles. As it stands they are better at it than defender missiles. Shouldn't smartbombs be for killing drones? Does anyone even use Smartbombs for killing missiles? I've seen in used in the alliance tournament years go, our team even used it, but it doesn't work, it doesn't stop ****.
Missiles seriously need to be removed from being effected by AOE weapons and replaced with something that doesn't kill the servers for the sake of a feature that is hit miss at best (as far it working to kill a missile in a laggy environment or not)
Maybe even simpler. Get the best of both worlds. It's a mid slot (or low?) module that targets a ship, and slows down it's rate of fire and/or missile speed (scripts).
Now you can "protect" your buddies from missiles, and with the removal of missile updates for everyone and not just the person being shot at.
Quote: You have to remember that missiles interact with AoE weapons. We have given this significant thought but it's not trivial to implement. You will note that even if the fighter bomber graphical effect is missile-like and even if the fighter bomber damage is calculated missile-style then you can't use AoE weapons to defend against their missiles.
BUT WHY do they interact with AOE weapons. Whats the design here to accomplish? I'm not saying it would be easy to do, but removing them gives you a lot more options.
What makes keeping missiles effected by AOE so important that it justifies the load it causes?
I'm not saying make them work like turrets, just make them local, so if you turn off effects it won't bother you.
I mean we already got drones. and defender missiles can't hit missiles shooting people other than yourself. So if the weapon put in the game to stop missiles can't target and defend your comrades, please just tell us *me* why smartbombs do?
|
Max Kolonko
Caldari Worm Nation Ash Alliance
|
Posted - 2010.12.12 20:02:00 -
[94]
Originally by: MotherMoon Edited by: MotherMoon on 12/12/2010 19:15:17
Originally by: CCP Explorer
Originally by: Siena Petrucis Usally, pilots in a big fleet fight have all weapon effects turned off. So that means there are close to zero observers for missiles. It would be a waste to send all the updates about created and destroyes missiles to clients which do not render it. Does the server take that into account? If not, would it be a possible way for more optimization?
All the clients in the bubble need to have the state of the simulation because they process it in sync with the server. There are also other reasons for the client to know about missiles than to render them, such as the interaction of missiles with AoE weapons.
ok lets be honest, It doesn't feel like an intended feature to have smartbombs randomly take down missiles. As it stands they are better at it than defender missiles. Shouldn't smartbombs be for killing drones? Does anyone even use Smartbombs for killing missiles? I've seen in used in the alliance tournament years go, our team even used it, but it doesn't work, it doesn't stop ****.
Missiles seriously need to be removed from being effected by AOE weapons and replaced with something that doesn't kill the servers for the sake of a feature that is hit miss at best (as far it working to kill a missile in a laggy environment or not)
Maybe even simpler. Get the best of both worlds. It's a mid slot (or low?) module that targets a ship, and slows down it's rate of fire and/or missile speed (scripts).
Now you can "protect" your buddies from missiles, and with the removal of missile updates for everyone and not just the person being shot at.
Quote: You have to remember that missiles interact with AoE weapons. We have given this significant thought but it's not trivial to implement. You will note that even if the fighter bomber graphical effect is missile-like and even if the fighter bomber damage is calculated missile-style then you can't use AoE weapons to defend against their missiles.
BUT WHY do they interact with AOE weapons. Whats the design here to accomplish? I'm not saying it would be easy to do, but removing them gives you a lot more options.
What makes keeping missiles effected by AOE so important that it justifies the load it causes?
I'm not saying make them work like turrets, just make them local, so if you turn off effects it won't bother you.
I mean we already got drones. and defender missiles can't hit missiles shooting people other than yourself. So if the weapon put in the game to stop missiles can't target and defend your comrades, please just tell us *me* why smartbombs do?
I agree. Frankly, I wasn't even aware that one can do that with AOE weapons like SB. If there are some very precise reason for this AOE-missile-affection-thingie to be there, please tell us, but if not: let's just get rid of it. Max Kolonko |
Princess Morenta
Minmatar Republic Military School
|
Posted - 2010.12.12 21:00:00 -
[95]
Not our fault you new guys don't know or read up on the mechanics..
Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Pre torp nerf :(
|
Clansworth
Good Rock Materials
|
Posted - 2010.12.12 21:26:00 -
[96]
For the love of god, could you PLEASE upgrade the graphics and sound instead of wasting time fighting LAGGZ? Intel/Nomad |
MotherMoon
Huang Yinglong
|
Posted - 2010.12.12 21:40:00 -
[97]
Originally by: Princess Morenta Not our fault you new guys don't know or read up on the mechanics..
Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Pre torp nerf :(
new? well EXCUSE me :P
I'm far from new lol. also torps can't be shot down by SM anymore. And you still could of one with defenders instead of they were useful. :P
r they could make it so smartbombs can't shoot down missiles that aren't shooting you.
Either way I've played a lot of eve, and smartbombs are not an effective counter, so lets get a real one, and decrease lag.
|
Chaos Incarnate
Faceless Logistics
|
Posted - 2010.12.13 01:54:00 -
[98]
firstly, excellent devblog duet. I was pretty awful at computer science in college, and it's been written with care enough so i have an idea what's going on, so props++ to you. Anyhow, couple questions:
1) Is the CPU the main bottleneck of server performance at the moment? Has it always been the main bottleneck of server performance, or is this something that has emerged recently (or perhaps gained in importance as other limits on server performance have been pushed back)?
The reason i ask is that i've seen a lot of lag over the years from various things, databases and proxy servers and stackless IOs, but apart from the drone nerf however many years ago there hasn't been much mention of CPU issues that I remember; at least not in terms of optimizations like these.
2) assuming that the CPU is the main bottleneck now...as you guys keep working on these code optimizations and the max stable fleet fight numbers increase, is there anything that you guys are concerned will emerge as a new limiting factor? or is everything else (hopefully) scalable enough that it's just a question of processing time?
oh, also, bonus question: You guys call them bubbles, we call them grids (presumably because we guess that they're cubic?). Are they actually shaped one way or another on the server side? _____________________ Horrors! Demons in the deep! |
Debir Achen
|
Posted - 2010.12.13 04:16:00 -
[99]
Originally by: Princess Morenta Not our fault you new guys don't know or read up on the mechanics..
Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Funny you should say that. I've never tried it (nor currently can afford to), but I had wondered whether it would be possible to protect against a Drake blob by sticking an 8-high-slot ship loaded entirely with smartbombs and cap boosts in front of your turret fleet, and ripple-fire the smartbombs every second or so. Assuming lag didn't get you, is this even practical? |
Liang Nuren
Parsec Flux War.Pigs.
|
Posted - 2010.12.13 05:23:00 -
[100]
Hey, pretty sweet dev blog there - all the new dev blogs are letting you guys get good practice. Soon, you will be holding seminars telling other game companies how to write dev blogs.
Seems like a good optimization to me. I really thought you guys were going to announce a Drake nerf with this dev blog series - fantastic job on fixing the problem instead of just nerfing the Drake.
-Liang -- Eve Forum ***** Extraordinaire On Twitter - Blog got deleted when Evepress died - |
|
Tric Starless
|
Posted - 2010.12.13 07:02:00 -
[101]
Great Dev Blog. Thanks.
Now having profiled/optimized a lot in the distant past... and knowing how programmers under deadline like to copy/paste code, I'm almost willing to bet a dollar that you've got that same (or similar) innefficient loop scattered throughout your codebase. You're probably already looking into that though aren't you.
Regards
|
Jorn Dwer
Gallente Center for Advanced Studies
|
Posted - 2010.12.13 07:06:00 -
[102]
Originally by: Debir Achen
Originally by: Princess Morenta Not our fault you new guys don't know or read up on the mechanics..
Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Funny you should say that. I've never tried it (nor currently can afford to), but I had wondered whether it would be possible to protect against a Drake blob by sticking an 8-high-slot ship loaded entirely with smartbombs and cap boosts in front of your turret fleet, and ripple-fire the smartbombs every second or so. Assuming lag didn't get you, is this even practical?
I'll just leave this here http://www.youtube.com/watch?v=1E9uVtP7IQ4&feature=related
|
Mr Garo
|
Posted - 2010.12.13 09:24:00 -
[103]
Thank you for an excellent dev blog. As a programmer I really appreciated the level of technical detail here. Keep these coming!
- Garo
|
Gretcherl
|
Posted - 2010.12.13 09:28:00 -
[104]
I'm really pleased with the optimisations. It also demonstrates well the attention to detail required in the programming profession.
I trust that CCP are making use of the more `conventional' optimisations, such as eliminating unnecessary computations and input/output, using intrinsic functions rather than loops, xrange() instead of range(), loading functions in local memory, etc.
:)
|
SirHarryPierce
|
Posted - 2010.12.13 13:31:00 -
[105]
great read! keep the graphs comming.
|
tehSiner
Abnormal Experience
|
Posted - 2010.12.13 14:29:00 -
[106]
since Evolve() is so "light" can we perhaps expect boxes in addition to spheres?
abnormal behavior of abnormal brain makes me normal |
Siena Petrucis
Caldari German Kings Majesta Empire
|
Posted - 2010.12.13 14:32:00 -
[107]
Originally by: Debir Achen
Originally by: Princess Morenta Not our fault you new guys don't know or read up on the mechanics..
Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Funny you should say that. I've never tried it (nor currently can afford to), but I had wondered whether it would be possible to protect against a Drake blob by sticking an 8-high-slot ship loaded entirely with smartbombs and cap boosts in front of your turret fleet, and ripple-fire the smartbombs every second or so. Assuming lag didn't get you, is this even practical?
Yes its possible. But there is virtually no practical use. The smartbomb ships need to be in exact line between the missile ships and the other ships they are protecting. They must keep a good formation without gaps and without much overlap (otherwise they kill each other). Its better to put these pilots into logistics or jammers - that will give you more and also more reliable damage mitigation.
|
Siena Petrucis
Caldari German Kings Majesta Empire
|
Posted - 2010.12.13 14:38:00 -
[108]
Originally by: CCP Explorer
Originally by: Siena Petrucis Usally, pilots in a big fleet fight have all weapon effects turned off. So that means there are close to zero observers for missiles. It would be a waste to send all the updates about created and destroyes missiles to clients which do not render it. Does the server take that into account? If not, would it be a possible way for more optimization?
All the clients in the bubble need to have the state of the simulation because they process it in sync with the server. There are also other reasons for the client to know about missiles than to render them, such as the interaction of missiles with AoE weapons.
Just remove that. As stated above, there is no impact on the useful gameplay.
I guess another problem is missile flight time.Remove that, too. Even with missile flight time eliminated, there are still working sufficiently different from turrets.
|
Lili Lu
|
Posted - 2010.12.13 15:46:00 -
[109]
Originally by: Jorn Dwer
Originally by: Debir Achen
Originally by: Princess Morenta Not our fault you new guys don't know or read up on the mechanics..
Personally fought 3 ravens and won with a CNR and 2x well timed faction smartbombs :p
Funny you should say that. I've never tried it (nor currently can afford to), but I had wondered whether it would be possible to protect against a Drake blob by sticking an 8-high-slot ship loaded entirely with smartbombs and cap boosts in front of your turret fleet, and ripple-fire the smartbombs every second or so. Assuming lag didn't get you, is this even practical?
I'll just leave this here http://www.youtube.com/watch?v=1E9uVtP7IQ4&feature=related
I'm so tired of people who had nothing to do with this trotting it out over and over as if it still works. I can criticize you for this because that's me sitting in the middle of that fleet. It worked pretty damn good the first time. However, there are lots of problems with getting it to work, addressing counters and such, which I don't want to go into here because it could still work for INIT again.
But, if you were folowing the INIT killboard over the weeks since that shiney video you'd notice that Firewall fleets dropped out of favor. And what is INIT flying now? . . Drakes So, please everyone who is tempted to do so stop being simpletons and posting this video as proof that Drakes don't need to be nerfed, or whatever you are posting it for. It was a fun moment in time in EVE tactical fleet development that may reemerge but for now is largely abandonned.
Nice blogs. Missiles deserve to be in the game, server CPU load or whatever. It's great you are fixing some of the problems with them. However, it still won't address the changes and other problems in game that have brought the Drake, a single BC out of 8 let alone 12 BSs, into the top spot in the serious fleet battle ship choice position. Even if you fix missiles here you will still have a Drake plague which might even get worse. So instead of Drakes figuring on 3 times the kills v the next closest competitor you will soon have 4 5 or 6x representation of Drakes over the next competitor. Other threads have discussed nerfing Drakes, which I hope you devs continued to read. Nerf the ****ing Drake as you have hinted at. You know you've got a problem with that ship as you jokingly included them in the title for these two blogs. Thanks. |
Thirler
The Arrow Project Morsus Mihi
|
Posted - 2010.12.13 16:36:00 -
[110]
Very good blog about optimization. I'd love to see even more of those.
|
|
Tres Farmer
Gallente Federation Intelligence Service
|
Posted - 2010.12.13 17:36:00 -
[111]
Very nice read. If there would be a vote for the Dev Blog of the year, this one would be my favourite! Keep em coming!!!!
support Public Idea Tracker | 24hr PLEX |
totalamd5
Goonswarm Federation
|
Posted - 2010.12.13 23:05:00 -
[112]
Edited by: totalamd5 on 13/12/2010 23:09:41 So... Why it starts to run much faster (more fps) after switching off brackets (not the effects)?
|
Cresalle
|
Posted - 2010.12.14 01:56:00 -
[113]
Missiles are 'items' because they behave like other 'items'. They have a location and a vector and a very simple AI for target tracking. Making them 'non-item' entities would have a greater cost than benefit. For instance, you can outrun a missile, depending on range, ship speed, missile speed, etc. You can smartbomb a missile. If CCP ever decides to fix them, you can use defenders against a missile. Changing missiles to be 'non-item' entities would only be a noticable improvement in situations where the missile lag is creating a significant percentage of the lag. Even in such cases the improvement would only be a percentage of the percentage of cpu load caused by the missiles. The change being proposed would require a lot of work on the part of the devs that could be better spent on other things. At the moment I really think drones deserve a LOT more attention than missiles since a couple supercaps can quickly drop a node's performance into the toilet.
As far as this fix, I'm glad it was done. To be completely honest quadratic loops would be the very first thing I'd be looking at (with a very large microscope) if I were working on optimizing this kind of thing.
There is one thing that I cannot let stand, and that is the facetious comment about python vs cpp. Saying that you can write crappy code in cpp is no kind of justification for using a less-optimal language. I'm sure you already know this, but allow me to make a gratuitously graphic allegory.
Common sense: Eating apples is better than eating poop. CCP: No way. Common sense: Yes. CCP: No, because if the apple is rotten then you can get sick. Common sense: *old-fashioned look*
|
Debir Achen
|
Posted - 2010.12.14 05:56:00 -
[114]
Originally by: Cresalle There is one thing that I cannot let stand, and that is the facetious comment about python vs cpp. Saying that you can write crappy code in cpp is no kind of justification for using a less-optimal language.
Depends on where you're taking the speed hit. If you spend lots of time context switching across syscalls or blocking on DB access, then low-level vs high-level won't make much difference. And don't forget that while an interpreted language will be slow the first time the code is executed, running the same code thousands of times quickly removes the compile hit.
It's also possible to write more complicated code better in a higher level language, since you can reach a higher level of abstraction without overflowing your cognition with mundane implementation details.
Sometimes, a higher-level compiler can do better than hand-tuning over a larger problem space. And sometimes it can't. But "use a lower-level language" should normally be a last resort for hardware-specific optimisation. As a general solution, it might actually make things worse. |
Iamien
Democracy of Klingon Brothers R.A.G.E
|
Posted - 2010.12.14 13:04:00 -
[115]
Originally by: CCP Masterplan
Secondly, when it is laid out in several lines of psuedocode, it becomes very easy to see what is wrong. However those few lines represent many hundres of lines of real code, spread across multiple modules, that have evolved over time to do many things. It was quite a lot of work to understand the low-level elements, and then to see how they piece together, finally forming the full picture. It is entirely possible that initially it was written in a different way altogether. Piece-by-piece it was changed over time, eventually arriving at where it was before I began looking into it.
Does CCP still employ any of the original developers responsible for any of the crucial low-level code? Even if they moved on after release to different projects, you would think after EVE started to see success that an effort would be made to re-acquire some of these people as they would provide invaluable insight on how stuff was put together. Every programmer has a different style and without vigorous documentation, you are in for a lot of agonizing work finding what your code does.
|
De'Veldrin
Minmatar Green-Core The Obsidian Legion
|
Posted - 2010.12.14 14:42:00 -
[116]
My favorite part is this:
Originally by: CCP Masterplan graph
Old :( New :)
Confirming once and for all that New > Old.
Also, loved the info on Destiny. Awesome sauce for Team Gridlock. --Vel
I'm more of a care-badger. |
Connor Kent
Caldari Icelandic Oppression Haven
|
Posted - 2010.12.14 19:21:00 -
[117]
So, what happens when you add enough ships & observers to make the CPU peg at 95% again? What's the new bottleneck?
|
Vincent Athena
|
Posted - 2010.12.14 20:02:00 -
[118]
Edited by: Vincent Athena on 14/12/2010 20:03:19 Fun read! Nice work.
I'm wondering if it would work to have every missile launcher come with a pre-allocated ball for its missile, rather than creating and destroying the ball every time the missile fires. The clients would all know that this ball would jump back to the firing ship once the missile hits or is destroyed. Maybe less cpu would be needed to handle missiles.
Longer time ticks: Two different methods have been suggested, and CCP has responded to only one (unless I missed something).
Method 1: do 1 second of simulation in 2 seconds, time slows. CCP response is we are considering it, there are issues with having 2 times scales going on at once.
Method 2: do 2 seconds of simulation in 2 seconds. Would this help in any way?
Also: you can view outstanding calls by hitting shift control alt m. Maybe this number should automatically be displayed somewhere, like above the time, whenever it is greater than zero for more than a few seconds. It would tell the players that things are getting laggy, stop pounding the keyboard and give the server a chance to catch up.
Edit for cut paste oopsie
|
Mzee Machado
|
Posted - 2010.12.14 20:45:00 -
[119]
These blogs are great. It is a joy to see pseudocode after 20 years of no programming. Great solving skills applied. Maharaba |
Jim Luc
Caldari Rule of Five
|
Posted - 2010.12.15 01:01:00 -
[120]
CCP Masterplan, I'd love to learn more about how the graphics are decided upon, for the lasers & missiles - how much of each relies on the actual math and physics.
One of the reasons is because when firing a missile, it seems that all missiles connect with my ship even if it says it misses. Same goes for lasers. Shouldn't some missiles explode around the ship, and lasers overshoot the ship, noticeably missing the vessel? It would be more realistic looking.
Also, the points on the ship that the lasers & missiles hit all seem to be the same, ie not randomized. If you watch a Sansha battle, you can see that about 50 ships are all hitting the exact same point on a ship, and when I get hit by enemy missiles, it's all hitting the exact same location. How hard is it to modify the client to display misses and randomize the point on the ship that it hits? Thanks!
|
|
Jas Dor
Minmatar
|
Posted - 2010.12.15 04:22:00 -
[121]
Originally by: CCP Explorer
Originally by: galphi You could also replace the physical object missiles with the 'fake' missile effects used by the fighter-bombers. It's not like anyone ever uses defender missiles anyway
You have to remember that missiles interact with AoE weapons. We have given this significant thought but it's not trivial to implement.
You will note that even if the fighter bomber graphical effect is missile-like and even if the fighter bomber damage is calculated missile-style then you can't use AoE weapons to defend against their missiles.
Oh for the love of god, just figure a percentage change of the missile getting blown up if an AoE or defender missile system is active (and draw down defender ammo by time). There I just fixed both your problem AND defender missiles.
|
Dillon Arklight
Universal Army Ushra'Khan
|
Posted - 2010.12.15 04:27:00 -
[122]
I have to admit to not following everything (its 4:30 am) but what i did follow looked good. Thanks for the blog and the insight into Team Gridlock's fight against lag.
|
Tres Farmer
Gallente Federation Intelligence Service
|
Posted - 2010.12.15 12:12:00 -
[123]
Originally by: Vincent Athena I'm wondering if it would work to have every missile launcher come with a pre-allocated ball for its missile, rather than creating and destroying the ball every time the missile fires. The clients would all know that this ball would jump back to the firing ship once the missile hits or is destroyed. Maybe less cpu would be needed to handle missiles.
What happens when [flight time of missiles to target] is bigger than [launcher cycle time]?
I don't think it's needed or productive that we speculate about ways to get this part of the code faster without knowing how it really works.
support Public Idea Tracker | 24hr PLEX |
WisdomPanda
Gallente Oberon Incorporated Morsus Mihi
|
Posted - 2010.12.15 13:24:00 -
[124]
Awesome blog, just don't listen to the haters on the low level code thing. If Python (or any other language for that matter) helps you write more efficient code on different levels, run with it. Lets face it, if low level performance was the key to everything, we'd all still be hand writing some form of assembly. We'd also take 8 years to publish an expansion and require and entire dev team to baby sit the graphics department. (No offense graphic guys, but I doubt assembly is what you think of when you go to work in the morning.)
Although secretly, they're all just wishing they could have your job. (Personally, I'd never hire anyone who's first answer to a performance issue is "use a lower level language." Clearly, CCP agrees.) ----- Cheesecake, Natures ultimate weapon. |
Serious Masta
Caldari
|
Posted - 2010.12.15 15:41:00 -
[125]
i like'd to read the articles and wanna see more of them!
i'm a developer and the shower-moment is a well known situation :)
|
Espacio Cristo
|
Posted - 2010.12.15 16:43:00 -
[126]
Just a quick one of those in the shower ideas. I'm sure I overlooked something important or stupid.
Would it be possible to group the drones of the same type together into one ball as we do with missiles? So instead of each individual drone having it's own ball, there is one ball representing the actions of the group.
The issues I can imagine: * This assumes that the group has the same target. * This would assume that all the drones are at the same location at any point and have the same velocity, orbit, etc... * Drone hit points and calculating damage from AOE, which drone takes the hit? |
Vincent Athena
|
Posted - 2010.12.15 18:46:00 -
[127]
Originally by: Tres Farmer Edited by: Tres Farmer on 15/12/2010 12:19:27
Originally by: Vincent Athena I'm wondering if it would work to have every missile launcher come with a pre-allocated ball for its missile, rather than creating and destroying the ball every time the missile fires. The clients would all know that this ball would jump back to the firing ship once the missile hits or is destroyed. Maybe less cpu would be needed to handle missiles.
What happens when [flight time of missiles to target] is bigger than [launcher cycle time]?
Have more than one preallocated ball.
I don't think it's needed or productive that we speculate about ways to get this part of the code faster without knowing how it really works.
Originally by: Vincent Athena Longer time ticks: Two different methods have been suggested, and CCP has responded to only one (unless I missed something).
Method 1: do 1 second of simulation in 2 seconds, time slows. CCP response is we are considering it, there are issues with having 2 times scales going on at once.
Method 2: do 2 seconds of simulation in 2 seconds. Would this help in any way?
The Space Simulation doesn't seem to be the bottleneck at them moment. It's still waaaay behind the serialisation of the information to be send to the clients and even that one is now waaaay behind the part of the code that creates/destroys missiles.
I dont mean just the flight stuff, I mean everything that happens in that tick. Somewhere above CCP said that the number of people that can interact in a MMO is proportional to the tick length, so making it longer in high load situations seems like a possibility.
Originally by: Vincent Athena Also: you can view outstanding calls by hitting shift control alt m. Maybe this number should automatically be displayed somewhere, like above the time, whenever it is greater than zero for more than a few seconds. It would tell the players that things are getting laggy, stop pounding the keyboard and give the server a chance to catch up.
You naively assume that ALL players want the server to stay in a working condition to fight the battle...
Those who want to cause lag will do so anyway. At least this will give an easy tool for those who do not.
|
Karbon Dating
|
Posted - 2010.12.16 06:15:00 -
[128]
Whats the issue with moving the physics engine and the rest of the stuff that happens in a system to different nodes? The physics engine gets all the CPU power it could ever want and all that entensive work for the rest of the stuff that goes on in a system would have more processing power to work with.
|
Seripis Chiktor
Amarr Cypher Industries.
|
Posted - 2010.12.16 12:33:00 -
[129]
from a programmer and network sec guy i have always been impressed with the server setup for this game. The structure and seamless bridging between blades is outstanding that alone is a lovely design. Apparently it cant be to easy to do as blizzard still keeps its servers split.
My question is this. Often time i notice lag even in the remote area i reside in in devoid. I know it is high sec but the traffic there is normally less than more low sec and null sec.
Its the simple things that cause lag as well. Warping in on a POS. Or engaging rats in the belts. Often times its just loading my messages in game or skills. some days it takes 1 to 2 mins to load messages or more. Yes i keep them clean.
I know these things seem trivial but are on the list to be addressed?
Also from a client side view I have noticed connection brownouts. from my home I develop code for clients and web dev. so my internet connection is active but in respect to other clients I run often I notice that the connection to eve will fade of sorts. at first I thought it might be the connection on my end but after doing some watching I have noticed that it is indeed the communication between server and client. the connection its self is not lacking but the load of data being transfered fluctuates by a margin of up to 30%
ok im done now lol just some things i have noticed. Seripis Chiktor Cypher Industries |
Shadow Watchman
|
Posted - 2010.12.16 13:49:00 -
[130]
And what about millions of mission/belt/anomaly NPCs spewing billions of missiles every second ? I think it is introducing persistent anomalies in Dominion caused a terrible load on the server :)
|
|
Forest Hill
|
Posted - 2010.12.16 14:59:00 -
[131]
Very interesting blog, touches on what I sometimes do at my day job.. HP diagnostics and loadrunner produce about the same graphs. Good stuff, keep 'em coming!
|
Tiruriku
|
Posted - 2010.12.16 15:55:00 -
[132]
excellent blog series, thanks.
|
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.16 16:44:00 -
[133]
Originally by: Vincent Athena Edited by: Vincent Athena on 14/12/2010 20:03:19 Fun read! Nice work.
I'm wondering if it would work to have every missile launcher come with a pre-allocated ball for its missile, rather than creating and destroying the ball every time the missile fires. The clients would all know that this ball would jump back to the firing ship once the missile hits or is destroyed. Maybe less cpu would be needed to handle missiles.
Longer time ticks: Two different methods have been suggested, and CCP has responded to only one (unless I missed something).
Method 1: do 1 second of simulation in 2 seconds, time slows. CCP response is we are considering it, there are issues with having 2 times scales going on at once.
Method 2: do 2 seconds of simulation in 2 seconds. Would this help in any way?
Also: you can view outstanding calls by hitting shift control alt m. Maybe this number should automatically be displayed somewhere, like above the time, whenever it is greater than zero for more than a few seconds. It would tell the players that things are getting laggy, stop pounding the keyboard and give the server a chance to catch up.
Edit for cut paste oopsie
If we pre-allocated balls for missiles, we'd still have to tell all the clients that a pre-allocated missile has now changed from 'in the launcher' to 'flying in space'. Figuring out which clients to send this update to would probably still need a similar amount of work.
Regarding method 2: This is one of the ideas we've considered. It would help reduce the proportion of time spent in Evolve (see the 'Destiny - Physics' area in blog 2, figure 1) However it wouldn't help the other areas of Destiny, as their cost is mainly per observer/missile. rather than per tick. There might be other issues with movements of very fast ships, for example when you can cover a large distance, or possibly even entire orbits within a tick duration.
You make a good point about the client providing some automatic display when server conditions are bad. Not everyone runs with the monitor window active. -- Team Gridlock: Herding electronic cats since 2010 |
|
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.16 17:01:00 -
[134]
Originally by: Jim Luc CCP Masterplan, I'd love to learn more about how the graphics are decided upon, for the lasers & missiles - how much of each relies on the actual math and physics.
Unfortunately I'm not the best one to answer this - client-side graphical effects aren't an area I've done much work with. Server-side, all we care about is Ship A firing at Ship B - we don't track it at any more detail that that. I agree that it would look great if we could do a lot of what you described. The only thing I can think of is that we don't send your client the hit/miss results of other ships shooting each other. So if you are watching a pair of ships fighting each other, what you see wouldn't be an accurate portrayal of who is landing the most hits. As you might be able to imagine, sending out this info would cripple the server in ways that make me shudder! -- Team Gridlock: Herding electronic cats since 2010 |
|
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.16 17:09:00 -
[135]
Originally by: Espacio Cristo Just a quick one of those in the shower ideas. I'm sure I overlooked something important or stupid.
Would it be possible to group the drones of the same type together into one ball as we do with missiles? So instead of each individual drone having it's own ball, there is one ball representing the actions of the group.
The issues I can imagine: * This assumes that the group has the same target. * This would assume that all the drones are at the same location at any point and have the same velocity, orbit, etc... * Drone hit points and calculating damage from AOE, which drone takes the hit?
Drone grouping is a regular topic here. You've picked up on some of the areas we'd need to address. None are impossible to solve - this is on our list somewhere. -- Team Gridlock: Herding electronic cats since 2010 |
|
BOOYA MOFO
|
Posted - 2010.12.16 17:14:00 -
[136]
Translate pls.
what happens to drakes?
|
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.16 18:31:00 -
[137]
Edited by: Daneel Trevize on 16/12/2010 18:33:13
Quote: If we pre-allocated balls for missiles, we'd still have to tell all the clients that a pre-allocated missile has now changed from 'in the launcher' to 'flying in space'. Figuring out which clients to send this update to would probably still need a similar amount of work.
Sure but wasn't it said that generating and removing the balls was the big hit? So if you had a steady stream of balls being generated (until a ball pool is full), then you're spreading this work out over time, while updating their state to now being out & about has less of an impact?
Equally, dead balls perhaps don't need to be fully retired & removed asap, but I guess this all comes down to how the interaction calculations & data structures can efficiently cope with such ball states being scattered throughout. |
Ebisu Kami
|
Posted - 2010.12.16 20:35:00 -
[138]
Edited by: Ebisu Kami on 16/12/2010 20:36:24
Originally by: Daneel Trevize Sure but wasn't it said that generating and removing the balls was the big hit? So if you had a steady stream of balls being generated (until a ball pool is full), then you're spreading this work out over time, while updating their state to now being out & about has less of an impact?
You want to create more balls in advance then needed? Strikes me as a bad idea. Really, your system does the same as the old, except for creating balls in advance, which will clog up the system additionally, while during a fight still working like the old system. That's not going to increase performance and will lower it eventually even more because of the extra-balls that need to be tracked.
|
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.16 21:08:00 -
[139]
Edited by: Daneel Trevize on 16/12/2010 21:10:21 Just trying to average out the ball creation. Perhaps there's some relationship between players on grid or launchers fitted and the average number of balls needing to be created & destroyed as a fight progresses, and that pre-creating several balls when breathing room is detected might keep the thing treading water for longer.
Again, unless I'm misremembering, tracking the balls isn't the issue so much as the creating & informing everyone with the related info, and having to do this a lot in too short a time. |
Ebisu Kami
|
Posted - 2010.12.16 22:15:00 -
[140]
Edited by: Ebisu Kami on 16/12/2010 22:25:59
Originally by: Daneel Trevize Just trying to average out the ball creation. Perhaps there's some relationship between players on grid or launchers fitted and the average number of balls needing to be created & destroyed as a fight progresses, and that pre-creating several balls when breathing room is detected might keep the thing treading water for longer.
You can't really average out engagement patterns, because a single fight is made up of a wide array of different paramters. Number of ships in fight, what is primaried, how the ships are fit, how the people decide to move, how far the ships are from each other, ammunition-type and and and and and and and and and and and. It's like trying to predict how each single hair looks after having a wild night. Even if you try to narrow it down somehow, you'll still not cover nearly enough to justify doing the statistics for a number of certain cases and the programming effort ultimately involved.
Originally by: Daneel Trevize Again, unless I'm misremembering, tracking the balls isn't the issue so much as the creating & informing everyone with the related info, and having to do this a lot in too short a time.
Tracking one ball might not be an issue, but multiply it with 100 ships. On both sides. And you also want to have balls stored in advance, so multiply it with the factor of your liking. Let's say a single missile-ship creates 40 balls in an average fight (although that's not going to work, as stated above), so you'd want to have that many balls per ship in advance, else your system obsoletes itself: 100*2*40=8000 balls that you need to keep track of + the ship-balls. You'll quickly end up at a point, where the overhead of tracking x balls has about the same effect as simple creation-destruction, plus at some point your pre-stored balls will be used up and you'd definitly need to create as many more as balls got destroyed, else people will not be able to fire missiles because they used all their balls or you'd just end up at a point where the balls are again created and used as before that little stunt... That's not going to work well.
Even worse, when do you want to create those balls in advance? You'd have to create and destroy them whenever a ship undocks or jumps through holes or gates, even for a simple grid-switch (each warp, basically) afaik, and that is for each and every ship that has missiles fitted... That's creating a ridiculously massive and completely unneeded overhead for absolutely no purpose.
Originally by: Daneel Trevize Edit: Upon re-reading the blogs, I'm reminded it seems the issue is mainly determining which clients to update with what, but the pseudocode and text seems to find this condition a key part of it: 'if ball was added/deleted to the bubble in the last tick'
It still seems there'd be a big payoff if you only had to track 'max-possible-missiles-in-space-from-a-given-launcher-including-overheat-etc' for the battle lifetime of a launcher/group/ship rather than create and destroy a ball for each unit of ammo he chucks. Once you're a few seconds into the brawl and you've already created that many balls, now you're just having a conveyor belt of them with some state changes rather than churning out more and more. Could even get some cache benefits of reusing such items but that's very hard to say. E.g. after 20secs you'd have added 10 balls but that's it, creation/destruction has peaked and you're almost back to where you were before loadwise. Currently you may have only added as many balls but in the next 20s you'll do it all again and then again, load remains higher if create&destruction outcosts state management.
You're really starting to compare apples and oranges. Your system is not going to bring any performance increase, really, because you basically change a dynamic system with overhead-spikes to a system that still has those overhead-spikes and on top of that a lot more overhead during normal operations.
|
|
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.17 01:29:00 -
[141]
Edited by: Daneel Trevize on 17/12/2010 01:31:24 Apply some logic. Only create balls for launchers if the hosting ship actually has launchers, online, loaded, and a target locked (or there's viable targets on the grid for FoF/Defenders).
As for averaging, we're already targetting the worst case. If people are uncoordinated (& very mixed skills) there will be less synchronised firing and reloading, instead I think the load would be slightly reduced if everyone's doing their own thing. But if they're all said blob of drakes, all in range, all reloaded at the same time, we can expect a spike in the need for balls every time their launchers cycle.
Instead if we precreated even half the expected balls that would be inflight once these ships are in full swing, we could perhaps have moved some load a few seconds or more earlier to perhaps a time when less intensive actions are happening, such as locking, and not at the time when perhaps drones are being reordered about when the primary call goes out. CCP obviously are the ones with the tools and stats to see if there is something like this going on, where several kinda of actions are bunched, with an opportunity to precalculate/prepare for some likely required actions.
You show that there would have to be a large number of balls tracked. However atm there are probably already even more being handled. A HAM drake can pack, what 66 missiles per launcher? Even if grouped that 12200 balls being handed (just) by the current system for our hypothetical fleet fight before a reload. But if they can only get 10 in the air at a time, that's 100*2*10 = 2000, an order of magnitude less by what I'm describing. Even if you buffer 100% more balls hanging around due to precreation and slow cleanup, that's still exponentially less going about.
And the second important benefit is that once those 2000-4000 are created and being managed, the load's back down to just juggling most of them, not creating & destroying each in an endless stream. |
Ebisu Kami
|
Posted - 2010.12.17 02:17:00 -
[142]
Edited by: Ebisu Kami on 17/12/2010 02:21:55
Originally by: Daneel Trevize Apply some logic. Only create balls for launchers if the hosting ship actually has launchers, online, loaded, and a target locked (or there's viable targets on the grid for FoF/Defenders). Even have the count grow with time, add 1 per group/launcher per tick until a defined limit.
You didn't notice that your idea just got even worse, did you? Also, you didn't really notice what's going on in larger fights nowadays, did you? Well, let me show:
Some larger alliances actually ask their members to specifically train for a Drake nowadays, to use it in fleet engagements. The Drake is one of the most common ships used in fleet-PvP. Also, the Drake is not the only ship on the field that has missile launcher slots and actually uses them.
Also now you only want to create your balls-reservoir, when the ship actually engages/locks a target, like when it warps into an already occupied grid... together with it's fleet... a fleet that has a good chance to have a number of ships with mounted, activated and loaded launchers... engaging a hostile fleet, which will alos have a good number of missile-mounted ships... Think about that for a moment...
|
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.17 03:01:00 -
[143]
Edited by: Daneel Trevize on 17/12/2010 03:04:36 No, I've never been in a 0.0 alliance or even a large fight. More than 5 people is a blob to me. Apologies for not joining in this crap sounding broken lagfest where it's metagamed to lag to benefit the ones there first.
That said, the point of the ball pool and recycling is to reduce the churn. The prepopulation idea, assume now including the responsive growth per tick in effect, is an optional extra. If your two fleets clash and let rip straight away, you're just creating and using the balls that currently have to be dealt with in that initial period and this mechanism wouldn't add any more. But if they're waiting for others to load grid, or cynos/bubbles, or see if the hostiles are going to aggress or run, you have time when the servers aren't under the crushing load yet, so why not invest those idle cycles and bandwidth in preparing for the probable, rather than do nothing?
It's been repeatedly stated that some players will always try to bring lag-inducing numbers, because playercount relative to eve performance they always can in at least 1 place at a time, and even if they don't cripple the fight, outnumbering your opponent is advantageous currently anyway. To fix that you must change the game to reward dividing forces, e.g. to adjecent system at the same time, in order to succeed at the current objective. That or relatively punish blobs, e.g. reduced stacking-style damage, or skillpoints training speed, or whatever.
Game theory is vital here. Again it's known mechanisms like fixed local/grid counts would be abused by 1 side getting there first.
What's your suggestion, other than the status quo? |
Vincent Athena
|
Posted - 2010.12.17 23:57:00 -
[144]
Actually when I suggested pre-allocated missile balls I was thinking there would be just one (or a few for rapid firing, long duration missiles) per launcher, and they would be reused. Part of the equations of motion for a missile ball would be that it snaps back to the launcher upon missile destruction. So that info need not be sent to each client, they got it built in.
Also no need to create and destroy the balls (well, maybe when a launcher is re-loaded, because it could then be a different missile type). The server needs only process and send " missile launched" to each client, not "create ball with this location, speed, missile, etc" because the clients already know that.
Surely creating a ball and moving it around takes more work than just moving it around?
But maybe CCP knows of better, low hanging fruit to go after.
Anyway; CCP keep up the good work!
|
Kaya Divine
Gallente Kittens Factory
|
Posted - 2010.12.18 01:01:00 -
[145]
I'm not gonna say that while you are doing this you could add/draw some bloody launchers to ships which have fitted launchers.
Shoot your shot... |
Berikath
|
Posted - 2010.12.18 17:48:00 -
[146]
Fair warning- I R comp sci student, so I apologize if this is one of those "knows just enough to be dangerous/annoying" situations, but just curious about a few different things; Originally by: CCP Masterplan Drone grouping is a regular topic here. You've picked up on some of the areas we'd need to address. None are impossible to solve - this is on our list somewhere.
It seems like this would also remove the problem of drones set to "focus fire"... umm... not actually focusing fire when they switch targets. You said it is a regular topic and was on the list; is it just a comparatively small performance gain currently, so it hasn't been implemented yet?
Originally by: CCP Masterplan
Originally by: Salpun The new fight on lag page places the smooth fleet fight limit at 600. What it does not say is if that is on a reinforced blade or not. Could you tell us? And the limit on the other blade type? Also with these new changes can you push that higher or do you have to have an actual fight on TQ that is with in standerd ie no lag to justify moving the number to the right.
Those numbers are based on a reinforced node. On a regular node, if all the other systems are empty, it will be almost the same as a reinforced node. It is impossible to say what a regular node will support in normal operation, as it all depends on what is happening in the other systems it is currently hosting.
Do you have any data on how usage grows in non-reinforced nodes with fights in multiple areas? From the sound of it, the change that was implemented brings performance down from the realm of about O ( N^2 ) to O ( N ) (yeah, I know you said it was N +/* M, but number of balls would seem to be ON AVERAGE directly related to number of players). Does the server now behave better handling 1 600 player fight than it does handling 60 10-player fights (since it would seem to negate much of the benefit of group updates)?
Originally by: CCP Explorer
Originally by: Daneel Trevize
Originally by: Black Dranzer WHY WOULD YOU DO THAT?
Because premature optimization is evil, so first you get it to work, and then you have other things to do
Indeed. First you write readable code that works. Then, based on measurements, you optimise the parts that need to be optimised.
o.0 I always found it helped a lot to draw up a rough high-level design doc in order to figure out a good/the best way to implement something, and that it actually saved me time in the long run because it let me break a problem down into smaller, easier problems that each took much less time (I suppose that being the whole point of object-oriented design). Am I missing something?
(Not trying to criticize, just it sounds like a different methodology than I've encountered before. I'm interested to understand the theory to find out if there's anything I could learn from it)
**
My own question: It seems like sending info on position for each missile to each client... well, really puts much more load on the server than need be. Why not send each client a dumbed-down "This person shot a missile at that object", let the client handle rendering missile flight and such, do all the position calculations and such server-side, and only sending the end result to the client? Admittedly, it does seem kind of hack-ish and might not be desirable in low-load situations but it seems like having a stripped-down mode available to switch to in extended high-use situations might not be the worst thing ever.
Are you concerned about unpredictable behavior from having modes that "switch"? Having to maintain two separate sets of code? Do you see sufficient room for improvement in current code that you expect optimizations to allow for fights as large as you would expect to see on-server?
*** [ SIG] ***
Wish list for PI:
*One-click input routing *Copy product, inputs & outputs in factories *Launchpad upgrades: twice the space, twice the cost, half the hassle! [ /sig ] |
Daneel Trevize
Black Viper Nomads
|
Posted - 2010.12.18 18:45:00 -
[147]
To me, the premature optimization argument is as follows: E.g. you build a system to produce hash codes for objects (for hash table use), but you decide ahead of time that having the method recalculate the value on every call will be too inefficient so you decide you'll cache the result after the first call, and just ensure it's updated when the values of the object it's based upon change (e.g. in each set() methods).
Now you spend the time adding this slight complexity and you find a speed issue with your app, but if/when you benchmark & profile it, the worst patch isn't anywhere near this piece, or worse, your new data structures or algorithm are less efficient than a simpler method, perhaps due to unexpected physical cache interaction or other external factors.
So yes you could take each thing your code's doing and perhaps plan a more efficient & complex version from the beginning, but you could well be wasting time, overly complicating & limiting the implementation from the required specification, and even making your final code or algorithm cause even just 1 more cache miss per 50000 cycles, which could rival all otherwise speed improvements depending on if you're IO bound, etc. |
Highfield
Caldari I.M.M Initiative Mercenaries
|
Posted - 2010.12.19 11:24:00 -
[148]
Great work on the blogs CCP!
A question that sprang to my mind when I was reading all this: What would happen with the server if we decided to go head to head with the old-fashioned rusty RRBS or SniperBS fleets? All this repairing-to-code seems to apply to missile spam only, does it improve server behaviour with a lot of turretships as well?
|
BigSako
|
Posted - 2010.12.20 12:14:00 -
[149]
very good, very nice to read! Thx for the good work and please continue posting stuff like this ;)
|
Pilk
Evolution IT Alliance
|
Posted - 2010.12.22 16:20:00 -
[150]
Originally by: Gnulpie Edited by: Gnulpie on 12/12/2010 12:35:43 Good speedup, this is pretty impressive.
But why does the machoNet::SinglecastByClientID take that long? The Destiny::SendToClients without the SingleCast is only 7.4 ms! So the overhead is not in sorting out the data any more but in sending it. Is it possible to delegate this to a (highly) parallel worker thread which is doing all the work in the background (on other nodes maybe?) or do you get in trouble then with the non-blocking paradigm of Destiny?
Edit: Do you lookup always SessionID's from ClientID's in each SingleCast? That would be a huge waste in this case as they are almost always constant.
I would also propose that a multicast approach would be more appropriate, since a large number of clients are each receiving the same data. Obviously not IP multicast, but passing a piece of serialized data exactly once to the proxy layer, along with a list of clients which must receive it. If the concern is that people might be connected through different proxies, at least three possibilities suggest themselves:
- Randomly choose a proxy server; this Sol sends a multicast to it:
multicast("Ship #21 firing an EM smartbomb, ship #33 has stopped moving",[Joe,Billy,Bob,Sam,Reggie],master=true) . That proxy then sends to each other proxy: multicast("Ship #21 firing an EM smartbomb, ship #33 has stopped moving",[Joe,Billy,Bob,Sam,Reggie],master=false) (and parses and runs the multicast for its own connections). Implicitly, it is not an error, and is silently ignored, if a given recipient of a multicast is not connected to the proxy that received it.
- From the Sol, contact every proxy and perform the same steps as above. Has the advantage that it can (more easily) be blocking to whatever extent is deemed necessary.
- On the Sol, maintain a list of proxies through which players are connected; the list need not be minimal, just complete. As above, but only contact each proxy which actually has a player connected. This one scales better, and is my favorite of the three.
This closely mirrors a fundamental Google approach to cluster computingłnamely, that it is rarely correct to select something. As an example, for certain classes of boolean search problems, it is expensive to evaluate their class so as to know which of several different algorithms to use to solve them. What Google does instead is launch all three of those algorithms simultaneously on three different machines; the first result to return is used, and the other two tasks are killed. So, in our case, if serialization is expensive, and if finding out to which proxy a given player is connected soaks up time, don't bother doing it.
--P
Kosh: The avalanche has already started. It is too late for the pebbles to vote. Tyrrax's bet status: PAID! |
|
Ankbar
Bene Gesserit ChapterHouse Sanctuary Pact
|
Posted - 2010.12.22 18:12:00 -
[151]
yes, i've enjoyed the read.
|
Hashin Kyojin
|
Posted - 2010.12.25 00:07:00 -
[152]
I love how Masterplan is explain everything to us. Keep up the hard work CCP. I loved reading both of the blogs and looking forward to see more to come.
|
Killajoules
|
Posted - 2010.12.25 08:06:00 -
[153]
Appreciate the transparency and all the hard work. Truely I have never seen a company or their employees work so hard for their customers.
Keep up the great work, we really appreciate not only what you do, but the time you spend explaining to us what/how/when you did it and honestly answering the additional inquiries that follow.
Best holiday wishes to ccp and their families!
|
Erik Finnegan
Gallente Polytechnique Gallenteenne
|
Posted - 2010.12.25 09:12:00 -
[154]
Found the dev blog only now thanks to the newsletter giving me a review. And I love pseudo code ! Thanks for not withholding from us the omega calculus either. |
Erik Finnegan
Gallente Polytechnique Gallenteenne
|
Posted - 2010.12.25 09:21:00 -
[155]
Edited by: Erik Finnegan on 25/12/2010 09:21:32 oops...double post ?! |
|
CCP Masterplan
C C P Alliance
|
Posted - 2010.12.25 12:53:00 -
[156]
Thanks for the positive feedback, I'm happy you enjoyed it.
Merry Xmas folks - Don't each too much turkey! -- Team Gridlock: Herding electronic cats since 2010 |
|
Luckytania
Gallente Bullets of Justice
|
Posted - 2010.12.25 16:36:00 -
[157]
Originally by: Turix I must say, this is exactly the kind of information I love to read about.
Personally, being a software engineer myself, I do like the detail (Psuedo-Code + Graphs = Win!). Defiantly keep writing these types of blogs and if possible encourage other teams to do the same, with the same level of detail! It would be... awesome?
Me too. I second all the above.
(Although, it would be OK with me if you don't have to write these articles defiantly. But definitely please do give us more.) |
Sed Man
Gallente Deus Imperiosus Acies
|
Posted - 2010.12.28 04:37:00 -
[158]
One of the best, if not the best dev write up I have read.
Well done CCP for hiring this person.
|
Miyuki Yotaka
|
Posted - 2010.12.28 15:07:00 -
[159]
Thanks, this was a very interesting read combined with part 1. :) Very well explained and still detailed! Keep up the good work in crushing the lag!
|
Lord Raven011
|
Posted - 2010.12.30 07:47:00 -
[160]
good work on the blog its great to be able to get an insight into how the destiny engin is working behind the client.
keep up the good work ccp
Raven
|
|
Isabella Thresher
Trinity Rise United Front Alliance
|
Posted - 2010.12.30 16:48:00 -
[161]
i found that to be a highly interesting read. very detailed, well written, nice graphics.
would love to see more of those!
|
Abramul
|
Posted - 2010.12.31 18:55:00 -
[162]
Just now seeing this, but worth a shot asking;
What happens if you don't bother sending missile positions at all, and instead only send damage notifications to the attacker and the target?
|
Bob Fegaipeor
|
Posted - 2011.01.06 10:08:00 -
[163]
That was really cool and interesting please do more like this, It is really nice to see what really goes on behind the scenes, instead of just telling use that stuff is getting done :p this makes it more believable :D
|
Johiidk
|
Posted - 2011.01.06 14:14:00 -
[164]
This is one of the best dev blog's right up there next to server maintenance, and blown up servers.
|
RuairiEVE
|
Posted - 2011.01.09 17:42:00 -
[165]
Fascinating read. As a programmer myself, I enjoy problem solving like this. I have some questions though: 1) Does each player create a bubble around themselves? Do nearby players all form into one bubble thus all subscribing to the same events within that bubble. 2) from a network and serialization point of view, what gets sent during the lifetime of a missile? Surly everything is known right at the launch. 3) assuming everyone in close proximity uses the same bubble, why do things like missiles even have to be simulated as balls? Aren't they just a timed event from when they are launched? 4) how are inventory updates handled? Assuming, once again, it's a shared bubble; do ammunition counts get cached as part of that bubble and then committed upon leaving?
Keep up the great work guys, looking forward to reading more dev blogs.
|
Lerayah
|
Posted - 2011.01.15 17:39:00 -
[166]
Edited by: Lerayah on 15/01/2011 17:39:57 loved it, please continue on in-depth insights :)
also, i'd greatly appreciate to see some of the actual algorhythms in real code :)
|
|
|
|
Pages: 1 2 3 4 5 6 :: [one page] |