SRL Archive - MSI Suggestions

From later in the thread.. Seems to be a good direction to head right now.

From: Coh3n
I don't remember if I said this already, but here's another direction. I think this is the easiest to do, most organized and easiest to understand.

1. Locations, items, objects, animations, etc. are loaded/unloaded in the functions their used. That way when none are being used, no extra memory/CPU is being used.
2. Only scripts are loaded for each player, and the CurrentScript variable will serve more of a purpose. This way we have only that player's script loaded instead of all. Right now, that would be only one script until we have support for more.

Again, can't remember if I said that already, I can't be bothered to check.

• MSI_Scripts
This may involve more work than other arrays simply because this array is what MSI is revolved around.
• MSI_Locations
This shouldn't be worked on right now as NCDS and I are working on a completely new TLocation record. Once the new record is released, this global array can be made local.
• MSI_Items
• MSI_Spells
• MSI_Obstacles
• MSI_Animations
• MSI_Materials
• MSI_Objects
• MSI_Obstacles
• MSI_Extras and Tracking need to remain global to work properly I think.

Now that MSI is, well, huge, we have A LOT of global arrays that hold objects/items/locations/whatever in memory, 90% of which aren't used. In my opinion, this is bad as it uses a lot of unnecessary resources. If a user is autoing with one player with one script, why load every other script as well? No point really.

So what I was thinking (Nava suggested it when I was looking into the memory leaking), why not set the global arrays when needed (i.e. in player setup). We'll load just the script/locations/items/whatever, for just that player, and unload them when the player is finished. This will make MSI use less resources when it's running (maybe get rid of some Simba crashing, I dunno), and decrease setup time (which really isn't a big deal, but we shouldn't have to wait 3-5 seconds for MSI to setup).

The only thing I'm not sure about is exactly how it will be set up. To make the transition as smooth as possible, we can still set the length of every array to the total amount of scripts/locations/etc., but only set the indexes that will be used. This way, there should be virtually no changes throughout the include.

The other option would be to set the length each time we switch players, which would require a lot more thinking and changes in the include, and as far as I know, empty indexes in an array take up virtually no memory, so I don't think it's a big deal.

What do you thik?

P.S. I think the most significant difference will be the DTMs in TItems, but if we're to do one array this way, we should do them all.

I certainly think we should do something with it.

I can post a more detailed opinion in a little bit.

I don't see any downsides…

I'm curious about this part: " To make the transition as smooth as possible, we can still set the length of every array to the total amount of scripts/locations/etc., but only set the indexes that will be used. This way, there should be virtually no changes throughout the include." Why don't we want to make changes throughout the include? If we have to make changes, then I'd be happy to do it as long as the result is as good as possible

Empty indexes take up space, too ;)

From: Zyt3x
I don't see any downsides…

I'm curious about this part: " To make the transition as smooth as possible, we can still set the length of every array to the total amount of scripts/locations/etc., but only set the indexes that will be used. This way, there should be virtually no changes throughout the include." Why don't we want to make changes throughout the include? If we have to make changes, then I'd be happy to do it as long as the result is as good as possible

Because I'm not sure exactly how much would need to be changed, and how difficult it would be. Wasn't sure if it was worth the trouble.

From: nielsie95
Empty indexes take up space, too ;)

I know, but I wasn't sure how much; if it was significant enough to make a difference. I suppose it would still add up.

E: If we were to set the length depending on what each player needed, it wouldn't be as easy as just calling MSI_Locations[LOC_VE_BANK], because LOC_VE_BANK constant is set to some higher number than the length. You can say let the length like [LOC_VE_BANK..LOC_VE_MINE], but there would still be empty indexes in between, just probably not as many.

From: Coh3n
Because I'm not sure exactly how much would need to be changed, and how difficult it would be. Wasn't sure if it was worth the trouble.


I know, but I wasn't sure how much; if it was significant enough to make a difference. I suppose it would still add up.

E: If we were to set the length depending on what each player needed, it wouldn't be as easy as just calling MSI_Locations[LOC_VE_BANK], because LOC_VE_BANK constant is set to some higher number than the length. You can say let the length like [LOC_VE_BANK..LOC_VE_MINE], but there would still be empty indexes in between, just probably not as many.

Its the exact same..

Memory is allocated regardless if it is actually assigned.

From: Nava2
Its the exact same..

Memory is allocated regardless if it is actually assigned.

Well then we'll have to work something out so we don't have any empty indexes.

Came up with this. Not sure how practicable it is, though.

procedure MSI_SetGlobals();
begin
  SetLength(MSI_Scripts, 0);
  SetLength(MSI_Locations, 0);
  // etc., etc..

  with MSI_Players[CurrentPlayer] do
  begin
    for i := 0 to High(Scripts) do
    begin
      // MSI_Scripts
      SetLength(MSI_Scripts, Length(MSI_Scripts) + 1);
      MSI_Scripts[High(MSI_Scripts)] := MSI_GetScript(Scripts[i].Name);

      // MSI_Locations
      SetLength(MSI_Locations, Length(MSI_Locations) + 2);
      MSI_Locations[High(MSI_Locations) - 1] := MSI_GetLocation(Scripts[i].PathInfo.StartLoc);
      MSI_Locations[High(MSI_Locations)] := MSI_GetLocation(Scripts[i].PathInfo.StartLoc);

      // MSI_Items

      // MSI_Objects

      // MSI_Spells

      // MSI_Obstacles

      // MSI_Animations
    end;
  end;
end;

// Returns the index in the global array that matches 'Constant'
function MSI_GetArrayIndex(WhichArr: Integer; Constant: Integer): Integer;
var
  i: Integer;
begin
  case WhichArr of
    ARRAY_SCRIPT:
      for i := 0 to High(MSI_Scripts) do
        if (MSI_Scripts[i].Constant = Constant) then
        begin
          Result := i;
          Exit;
        end;

    ARRAY_LOCATIONS: begin end;
    ARRAY_ITEMS: begin end;
  end;
end;

We could just not use global arrays at all, and load the type we need in each function. That seems to be the most practicable solution. If we were to do that, the MSI_GetScript/Location/etc. functions would need to be changed to return that type, then all the functions/procedures would need to be edited (which isn't really that big of deal). I would, however, like to keep the files the way they are (there's no sense in opening/closing the debug file every 100ms).

Just for the record, you can hold over 250k integers in 1 MB of memory, so you'll have to think about if this is really worth the effort. As you said, it might be worth it for the DTMs (they do not take up their full space with an empty index), but it would only matter if you're talking about hundreds of DTMs (in my opinion). Of course it's good practice to only load the memory you're going to need, but it should also be worth the effort.

From: nielsie95
Just for the record, you can hold over 250k integers in 1 MB of memory, so you'll have to think about if this is really worth the effort. As you said, it might be worth it for the DTMs (they do not take up their full space with an empty index), but it would only matter if you're talking about hundreds of DTMs (in my opinion). Of course it's good practice to only load the memory you're going to need, but it should also be worth the effort.

^ Running SMART takes up way more.

Will it be illogical to load the arrays (as its already setup) call the constant for that particular player into a temporary record then free the array each time? This might take more cpu processing and compile time then its worth though..

From: nielsie95
Just for the record, you can hold over 250k integers in 1 MB of memory, so you'll have to think about if this is really worth the effort. As you said, it might be worth it for the DTMs (they do not take up their full space with an empty index), but it would only matter if you're talking about hundreds of DTMs (in my opinion). Of course it's good practice to only load the memory you're going to need, but it should also be worth the effort.

I think it would be worth the effort for DTMs for sure, as for the rest, I'm not sure. I mean, MSI is only going to expand, so it may be better in the long run if we were to switch now.

From: Narcle
^ Running SMART takes up way more.

Will it be illogical to load the arrays (as its already setup) call the constant for that particular player into a temporary record then free the array each time? This might take more cpu processing and compile time then its worth though..

That's kind of what I did above, except all the arrays weren't loaded at startup. The arrays were "reset" each time a new player began.

As Narcle suggested, a temporary record that is changed on player switching sounds like the most viable option in terms of memory management and implementation time-wise.

For loading the value you can retrieve them from functions, as you'd expect to get from a java .Get (except that here it is not an object's method but whatever)

Picture an MSI version of CurrentPlayer.

MSI_CurrentPlayer.Tools[best_tools].DTM
MSI_CurrentPlayer.Tools[best_tools].Level
MSI_CurrentPlayer.Tools[best_tools].Type
MSI_CurrentPlayer.Tools[best_tools].Material

this would then allow all those arrays to be managed from within one record, and therefore use up less space as you'd clear the record each time you change player.

There will most likely be some leaking for every load & unload, but shouldnt compare to the memory hog of keeping all DTMs on memory…

~RM

I like that idea, Narcle and Sir RM. Seems like the best way to do this (memory wise)

From: Sir R. M8gic1an
As Narcle suggested, a temporary record that is changed on player switching sounds like the most viable option in terms of memory management and implementation time-wise.

For loading the value you can retrieve them from functions, as you'd expect to get from a java .Get (except that here it is not an object's method but whatever)

Picture an MSI version of CurrentPlayer.

MSI_CurrentPlayer.Tools[best_tools].DTM
MSI_CurrentPlayer.Tools[best_tools].Level
MSI_CurrentPlayer.Tools[best_tools].Type
MSI_CurrentPlayer.Tools[best_tools].Material

this would then allow all those arrays to be managed from within one record, and therefore use up less space as you'd clear the record each time you change player.

There will most likely be some leaking for every load & unload, but shouldnt compare to the memory hog of keeping all DTMs on memory…

~RM

The trouble with that is the player won't always be using the best tool. They may be using a bronze pickaxe instead of steel or something like that, so not just the best tool's item record would need to be loaded. I'm sure something could be figured out though.



I dunno, I think we may be making it more complicated than it needs to be. nielise did say you can have 250k integers that take up only 1MB of memory (which really isn't anything), and all our global records (aside from MSI_Items) are integers/strings. So again, would it really be worth it to do this for all records? We're mainly concerned about the DTMs, so why not just load/unload the DTMs when switching players? I know I said the opposite on the first post, but things change. :p

You can use SizeOf(var) to check the size of a variable. For arrays (and strings) you'd have to multiply the element size with the array size.

program new;
var
  a: Integer;
  b: array[1..10] of Integer;
  c: array of Integer;
  d: record a, b: Integer; end;
  e: TMDTMPoint;
  f: array[1..15] of TMDTMPoint;
const
  mb = 1024 * 1024;
begin
  SetLength(c, 10);
  WriteLn('a = ' + IntToStr(SizeOf(a)) + ' -- ' + IntToStr(mb div SizeOf(a)) + ' in 1 MB');
  WriteLn('b = ' + IntToStr(SizeOf(b)) + ' -- ' + IntToStr(mb div SizeOf(b)) + ' in 1 MB');
  WriteLn('c = ' + IntToStr(SizeOf(c)) + ' -- ' + IntToStr(mb div SizeOf(c)) + ' in 1 MB');
  WriteLn('c = ' + IntToStr(SizeOf(c[0]) * Length(c)) + ' -- ' + IntToStr(mb div (SizeOf(c[0]) * Length(c))) + ' in 1 MB');
  WriteLn('d = ' + IntToStr(SizeOf(d)) + ' -- ' + IntToStr(mb div SizeOf(d)) + ' in 1 MB');
  WriteLn('e = ' + IntToStr(SizeOf(e)) + ' -- ' + IntToStr(mb div SizeOf(e)) + ' in 1 MB');
  WriteLn('f = ' + IntToStr(SizeOf(e)) + ' -- ' + IntToStr(mb div SizeOf(f)) + ' in 1 MB');
end.

So, yes, around 250k integers in 1MB. Around 3k DTMs if you say the average DTM has 15 points.

3k DTMS!? Jesus. I guess it's safe to say that our <50 don't make a difference? And all our DTMs have 5 points I think.

In case anyone was wondering:

Compiled succesfully in 15 ms.
a = 4 -- 262144 in 1 MB
b = 40 -- 26214 in 1 MB
c = 4 -- 262144 in 1 MB
c = 40 -- 26214 in 1 MB
d = 8 -- 131072 in 1 MB
e = 21 -- 49932 in 1 MB
f = 315 -- 3328   in 1 MB
Successfully executed.

MSI_Scripts = 14833 -- 378 in 1 MB
MSI_Scripts[SCRIPT_VE_MINER] = 163

MSI_Locations = 2772 -- 378 in 1 MB
MSI_Locations[LOC_VE_BANK] = 21

MSI_Objects = 11322 -- 92 in 1 MB
MSI_Objects[TREE_NORMAL] = 111

MSI_Items = 3654 -- 286 in 1 MB
MSI_Items[LOGS_NORMAL] = 21

MSI_Obstacles = 840 -- 1248 in 1 MB
MSI_Obstacles[OBS_LADDER_DOWN_MINE_GUILD] = 40

MSI_Animations = 672 -- 1560 in 1 MB
MSI_Animations[ANIM_ALCH_HIGH] = 32

MSI_Spells = 1932 -- 542 in 1 MB
MSI_Spells[SPELL_ALCH_HIGH] = 69

Does SizeOf return how many kB the variable is? E: Actually, that wouldn't make sense according to that output.

Haha, I don't think we'll surpass 14k scripts, EVER.

From: Zyt3x
Haha, I don't think we'll surpass 14k scripts, EVER.

That 14k is the size of the current MSI_Scripts array.

From: Coh3n
That 14k is the size of the current MSI_Scripts array.

Oh, right… Well we might surpass ~400 scripts in the future.

Coh3n… TOOL_BEST, TOOL_MAX(level dependent), TOOL_CURR, TOOL_LOW

:)

~RM

From: Sir R. M8gic1an
Coh3n… TOOL_BEST, TOOL_MAX(level dependent), TOOL_CURR, TOOL_LOW

:)

~RM

What is tool? Which of Cohen's posts are you referring to? Try not to be so vague in your posts, thanks.

From: Coh3n

Does SizeOf return how many kB the variable is? E: Actually, that wouldn't make sense according to that output.

SizeOf returns the number of bytes :)

From: lordsaturn
What is tool? Which of Cohen's posts are you referring to? Try not to be so vague in your posts, thanks.

the BLOCK CAPITALS signify it's a constant, as can be seen from MSI's several guides, as for tool,

TOOL_PICKAXE,
TOOL_HATCHET,
TOOL_FISHING or TOOL_HARPOON

whether the constant is exactly this one or not, I am unsure, although I mean the concept and I'm pretty sure the constant naming is very close to this one.

And I'm reffering to the posts in general concerning the loading and unloading of arrays (and suggesting the use of records), as well as reffering to the memory used.

The records reffer to the MSI v2 (internal version) thread, as my suggestion for v2 is to make things more record based, and (new suggestion) MSI_CurrentPlayer record.

procedure MSI_SetTools(activity: integer);
begin
  case activity of
    ACTIVITY_MINING: //use toolset IDs PICKAXE_BRONZE..PICKAXE_DRAGON
      //write function that returns best tool dependant on level
    begin
      MSI_CurrentPlayer.Tool[TOOL_BEST] := GetPickAxeLevel(MSI_CurrentPlayer.Level[Mining]);
      MSI_CurrentPlayer.Tool[TOOL_LOW] := MSI_AnalyseBank(TOOL_PICKAXE, false);
      MSI_CurrentPlayer.Tool[TOOL_CURR] := MSI_AnalyseBank(TOOL_PICKAXE, true);
      MSI_Withdraw(MSI_CurrentPlayer.Tool[TOOL_CURR];
  end;
end;

MSI_CurrentPlayer would be an extension of SRL's current player.
I'm unsure as to whether pascal script supports extensions, for which it might be easier to copy SRL's record, and use SRL's record to a minimum in order to keep all the info in one spot. I mean extend SRL's record, just not sure what would be the best way to go about it and exactly what it would involve

~RM

From: nielsie95

The records reffer to the MSI v2 (internal version) thread, as my suggestion for v2 is to make things more record based, and (new suggestion) MSI_CurrentPlayer record.

procedure MSI_SetTools(activity: integer);
begin
  case activity of
    ACTIVITY_MINING: //use toolset IDs PICKAXE_BRONZE..PICKAXE_DRAGON
      //write function that returns best tool dependant on level
    begin
      MSI_CurrentPlayer.Tool[TOOL_BEST] := GetPickAxeLevel(MSI_CurrentPlayer.Level[Mining]);
      MSI_CurrentPlayer.Tool[TOOL_LOW] := MSI_AnalyseBank(TOOL_PICKAXE, false);
      MSI_CurrentPlayer.Tool[TOOL_CURR] := MSI_AnalyseBank(TOOL_PICKAXE, true);
      MSI_Withdraw(MSI_CurrentPlayer.Tool[TOOL_CURR];
  end;
end;

MSI_CurrentPlayer would be an extension of SRL's current player.
I'm unsure as to whether pascal script supports extensions, for which it might be easier to copy SRL's record, and use SRL's record to a minimum in order to keep all the info in one spot. I mean extend SRL's record, just not sure what would be the best way to go about it and exactly what it would involve

~RM

I like the idea if I understand it correctly. Seems to make everything a little more organized, anyway, but I don't really see how it has anything to do with memory management. I also don't really know what the use of such a function would be other than during player setup. I mean, we'd be getting different tool stuff, but it wouldn't be used anywhere other than during player setup. And if by extension you mean inheritance, then I'm pretty sure pascal doesn't have it.

From: Coh3n
I like the idea if I understand it correctly. Seems to make everything a little more organized, anyway, but I don't really see how it has anything to do with memory management. I also don't really know what the use of such a function would be other than during player setup. I mean, we'd be getting different tool stuff, but it wouldn't be used anywhere other than during player setup. And if by extension you mean inheritance, then I'm pretty sure pascal doesn't have it.

I did mean inheritance,

got messed up in "MSI_CurrentPlayer extends SRL_CurrentPlayer" (java style)

as for the function it could be used it setup and it could also be used as the player levels up, eg if you are using a bronze axe but the best axe you have is actually mith, once the player reaches level 21 the record (and therefore the function) could be used to switch to a better axe

as for memory management I just meant the record, as instead of loading all dtms, you'd just load and unload the dtms for the current player ;)

~RM

From: Sir R. M8gic1an
I did mean inheritance,

got messed up in "MSI_CurrentPlayer extends SRL_CurrentPlayer" (java style)

as for the function it could be used it setup and it could also be used as the player levels up, eg if you are using a bronze axe but the best axe you have is actually mith, once the player reaches level 21 the record (and therefore the function) could be used to switch to a better axe

as for memory management I just meant the record, as instead of loading all dtms, you'd just load and unload the dtms for the current player ;)

~RM

Oh I understand now. :)

I like it.

However, in order to know which is the best hatchet the player has, we'd need to load the DTMs for all the hatchets, but I guess that could be done locally.

I don't remember if I said this already, but here's another direction. I think this is the easiest to do, most organized and easiest to understand.

1. Locations, items, objects, animations, etc. are loaded/unloaded in the functions their used. That way when none are being used, no extra memory/CPU is being used.
2. Only scripts are loaded for each player, and the CurrentScript variable will serve more of a purpose. This way we have only that player's script loaded instead of all. Right now, that would be only one script until we have support for more.

Again, can't remember if I said that already, I can't be bothered to check.

From: Coh3n
I don't remember if I said this already, but here's another direction. I think this is the easiest to do, most organized and easiest to understand.

1. Locations, items, objects, animations, etc. are loaded/unloaded in the functions their used. That way when none are being used, no extra memory/CPU is being used.
2. Only scripts are loaded for each player, and the CurrentScript variable will serve more of a purpose. This way we have only that player's script loaded instead of all. Right now, that would be only one script until we have support for more.

Again, can't remember if I said that already, I can't be bothered to check.

I think that sounds good. I don't see a reason why we shouldn't at least try it out. Might as well just do it when we go through all the code for MSI.

Okay great. I'm also discovering a lot about our object finding. I'm still doing some testing but using that alternative WaitWhileChopping and removing object tracking significantly decreases the amount of memory added.

So I'm not exactly sure what in those procedures could cause this, but I'll post it in this thread.

Bumping this. I've slowly been converting the records to be loaded locally. I've added a list to the first post.