Mapbase MP Port 2025 (Mapbase TF2)

[Blixibon]

This branch represents a minimum viable product for Mapbase running on the updated Source 2013 Multiplayer branch.

The current goal of this branch is to support Mapbase on Team Fortress 2. Preliminary support for HL2:DM and singleplayer HL2 also exist, although the latter is still not viable on this branch due to engine issues.

This branch is being developed in cooperation with the Team Fortress 2: Classic team, and many "generic" fixes or improvements from Team Fortress 2: Classic will be added to this branch in the future.

---

This branch has the following limitations:

• Mapbase's shader changes (parallax corrected cubemaps, projected texture fixes, etc.) have not yet been implemented.
• The VScript in this branch is functional, but it is in very bad shape internally and needs work. It may also not yet support everything TF2 supports.
• Mapbase's build automation scripts have not yet been adapted to the new dependencies.

Feature removals

Before using this branch, be aware that we intend to remove features from the game that are barriers to modding and/or that we are reasonably certain will not (or cannot) be used by mods.

Most of these removals were originally meant for Team Fortress 2: Classic in particular, but many of them are being upstreamed to Mapbase due to an overlap of work with other TF2 mods. We are still working out the details of exactly what would be necessary to remove, but for now, this is what's on the chopping block:

• The coaching system
• Free/trial account checks
• EOTL duck related features (duck still exists)
• TF2 achievements
• Promotional codes
• Main menu "training" popups
• Casual and competitive match type checks
• Matchmaking dashboard and associated panels
• Badge, PvP rank, and comp match summary panel
• tf_wearable_campaign_item
• Some fields for CEconItemDescription
• Contracker panels and quests
• Store/ad panels
• Vast majority of econ item tool code
• YouTube and twitch.tv integration
• Campaign medals
• Trading

If you are developing a TF2 mod which will rely on one or more of these features, please let us know.

Mapbase VScript + TF2 VScript

Mapbase and the TF2 SDK have their own separate implementations of VScript. The ideal state of Mapbase VScript in this branch would be a hybrid that supports everything TF2 VScript supports. Since they both derive from Alien Swarm, they have a common baseline, but they have also gone through several years of independent and sometimes convergent changes. As a result, these implementations still collide catastrophically overall and have serious conflicts.

There are three main points of complication:

• TF2 VScript has an overhauled script variant system which frees memory differently. Some of these changes conflict with fixes in Mapbase which resolved memory leaks in other ways.
• TF2 VScript has its own ways of registering and tracking script descriptions which partially diverge from Mapbase.
• Mapbase VScript has several features that were inspired by features in later Source games, such as the NetProps or Convars singletons. As the new branch contains much of the original code for these features, Mapbase now awkwardly contains both their original implementations as well as the independent implementations based on them.

For the time being, I have nested most conflicting sections of code from TF2 VScript in #ifndef MAPBASE_VSCRIPT and merged sections where I'm reasonably certain the functionality is identical, although I did have to forego a number of Mapbase changes (e.g. some of the new memory safety fixes for Vectors/script variants) because I wasn't sure how to merge or #ifdef them without risking further issues.

Many of the sections I wasn't sure about merging involved changes from @samisalreadytaken and @z33ky, who may have a better impression of how their features could coexist with TF2 VScript.

TF2 vs. Mapbase syntax

Some functions in Mapbase and TF2 have the same name, but different syntax. I've added a preprocessor to prioritize TF2 VScript's syntax when playing MP games (TF2 or HL2:DM) while prioritizing Mapbase's syntax in SP games (HL2).

Currently, singletons such as NetProps and Convars which exist in both TF2 and Mapbase will use Mapbase's version. These singletons still have some syntax differences which may not yet be compatible with all TF2 vscripts.

Other issues

• Weapon models on other players may occasionally disappear.
• In TF2, certain HUD effects (e.g. the timer above dropped intel) may flicker while cloaked as the spy.
• Ropes may not be visible at the moment. (I need to look into the rope code)

Future features

Beyond the base feature set, we hope what Mapbase does for TF2 will eventually be on a similar level to what Mapbase does for HL2. We have discussed adding better support for custom weapons and optional support for more than 2 teams, like what TF2:C has.

I also intend to add some features to HL2:DM which I was working on prior to the SDK update, including integration of Mapbase's upcoming protagonist and animation state systems, although I will focus on those at a later time (and potentially in a separate pull request).

How to use this branch

I do not currently have a build prepared, although if you are developing a TF2 mod using the SDK code, you should be able to merge this branch and use it as-is. We recommend using Git so that further changes can be easily pulled in the future.

The shared content VPK normally used in Mapbase mods has some scripts which weren't meant to be used with TF2, so it should not override TF2's file paths. Instead, you can put it in between TF2 and HL2 content, like this:

			game_lv				|appid_440|tf/tf2_lv.vpk
			game+mod			|appid_440|tf/tf2_textures.vpk
			game+mod			|appid_440|tf/tf2_sound_vo_english.vpk
			game+mod			|appid_440|tf/tf2_sound_misc.vpk
			game+mod+vgui		|appid_440|tf/tf2_misc.vpk

+			game+mod			|gameinfo_path|../mapbase_shared/shared_misc
+			gamebin				|gameinfo_path|../mapbase_shared/shared_misc/bin
+			game+mod+vgui			|gameinfo_path|../mapbase_shared/shared_content_v7_0.vpk

			game				|appid_243750|hl2/hl2_textures.vpk
			game				|appid_243750|hl2/hl2_sound_vo_english.vpk
			game				|appid_243750|hl2/hl2_sound_misc.vpk
			game+vgui			|appid_243750|hl2/hl2_misc.vpk
			platform+vgui		|appid_243750|platform/platform_misc.vpk

Note that this file structure will likely change for any eventual build release.

You can also choose to forego the Mapbase VPK entirely, although entity icons/dev texture assets and some specific Mapbase features (e.g. map-specific files, console groups) may not be fully functional.

Mapbase FGDs for Source 2013 Multiplayer (including TF2) can be found here: https://github.com/mapbase-source/source-fgds/tree/mapbase-mp

[z33ky]

Many of the sections I wasn't sure about merging involved changes from @samisalreadytaken and @z33ky, who may have a better impression of how their features could coexist with TF2 VScript.

I've taken an initial look at the changes.
#320 is still relevant (if SCRIPTHOOKs are kept) since it is about the interaction between SCRIPTDESC and SCRIPTHOOK, the latter of which doesn't seem to exist in the TF2 source. From a glance, it seems the changes were adopted correctly.

About #321, what's interesting (to me anyways) is that construction from Vector now always copies, disregarding the bCopy parameter. QAngle should be changed to behave the same. The assignment operators also copy in the new code, whereas they didn't before.
This kinda an alternative to providing the move constructors and assignment operators, since these are the ones to unconditionally copy in the "mapbase SP" code to prevent referencing dangling Vector/QAngles. It's a fair solution, I even proposed this in the original PR comment. It could be changed to the PR solution though, perhaps microoptimizing some Vector and QAngle assignments.
This does make the ScriptVariantTemporaryStorage_t optimization impossible I believe (the impact of which was discussed (for 32-bit archs...) in the PR). Perhaps it would be fair for the pointer assignment to not copy (like the pointer construction also allows). The assignment from the temporary storage object here and here then need to be changed to assign the address of the respective temporary storage field instead (i.e. add an &). It would maybe be wiser to add a separate function instead of changing the semantics of the existing assignment operator, for compatibility reasons?
The malloc()/free()/new situations looks to be fixed upstream already.
It's neat they changed Vector to be trivial, so ScriptVariant_t::EmplaceAllocedVector() for extra correctness points is no longer necessary. You did not copy the function to the new code, so that's fine, just CreateParamCheck() in vscript_squirrel.cpp needs to be adapted; My browser/GitHub doesn't deal well with a diff that large, so have a diff instead of a suggestion comment.

diff --git a/src/vscript/vscript_squirrel.cpp b/src/vscript/vscript_squirrel.cpp
index 0c5fa2fe6..0562120fb 100644
--- a/src/vscript/vscript_squirrel.cpp
+++ b/src/vscript/vscript_squirrel.cpp
@@ -1320,10 +1320,7 @@ bool getVariant(HSQUIRRELVM vm, SQInteger idx, ScriptVariant_t& variant)
 			tag == TYPETAG_VECTOR &&
 			SQ_SUCCEEDED(sq_getinstanceup(vm, idx, (SQUserPointer*)&v, TYPETAG_VECTOR)))
 		{
-			variant.Free();
-			variant = (Vector*)malloc(sizeof(Vector));
-			//variant.EmplaceAllocedVector(*v);
-			variant.m_flags |= SV_FREE;
+			variant = *v;
 			return true;
 		}
 		// fall-through for non-vector

FIELD_QANGLE no longer piggybacks on FIELD_VECTOR, which would enable a few cleanups in the SquirrelVM that treats QAngles as Vectors. This might mean we'll also want a dedicated SQQAngle in addition to SQVector? Anyway, this can (and probably should) be a separate change.

[samisalreadytaken]

My observations on vscript differences:

There is no native function in TF2 that returns new script instances such as CreateDamageInfo which required HSCRIPT_RC memory management.

IScriptVM::CopyObject from mapbase is called IScriptVM::ReferenceScope in TF2, which is a misnomer because the object doesn't need to be a scope. Manual calls to CopyObject are replaced with (unused) CScriptAutoRef abstraction. Programmers normally only need to understand that HSCRIPT is a script object and it can be copied or released; this abstraction feels like extra work.

They have ISquirrelMetamethodDelegate, but it's unreferenced in the SDK. It sounds like IScriptInstanceHelper::{Get,Set} in mapbase which is used by CAnimEventTInstanceHelper.

CScriptKeyValues is as broken as it always was, fixed in mapbase.

CScriptConvarAccessor functions return ScriptVariant_t to be able to return null instead of the zero value of the return type. This is a design choice, though I prefer not returning different types. I guess the benefit of this would be detecting if a cvar is blocked programmatically, but the downsides of user code needing to check every cvar is too much.

CScriptConvarAccessor::SetValue uses a whitelist in TF2, mapbase uses a blacklist.

CVScriptGameEventListener is the same as L4D2, it only executes a script function which then iterates through collected user functions and executes them in script. Mapbase saves user functions in C++, and directly executes them in the event call. L4D2 style is error prone and results in users always resetting events globally. I had to overwrite ClearGameEventCallbacks and GameEventCallbacks in L4D2 to guard my addon against third party addons.

The offset hack in mapbase CScriptGameEventListener is replaced with IGameEvent::GetDataKeys as was mentioned in mapbase.

Then there is "script events" which is quite weird. It saves script functions in script, then executes those script functions through script calls to C++. I suppose whoever implemented it in L4D2 wanted a unified way of calling script functions by different authors, but the C++ section of this is completely passthrough.

ScriptFireGameEvent doesn't actually fire the game event but calls script functions. Mapbase fires the game event as it should, like in Source 2. To remedy this, they seem to have added ScriptSendGlobalGameEvent which leaks memory.

It looks like CNetPropManager was contributed by L4D2 modders. It has new functions: get/set bool, GetTable, GetPropInfo. They seem pointless to me but they can be reimplemented.

They allow specifying SendTable or DataMap to differentiate between datamaps and netprops, but mapbase implementation of CScriptNetPropManager is overall more extensive. Performance wise, TF2's is about 40% slower than mapbase's. It gets even slower when sub-members are accessed.

Numbers roughly look like this:

// "m_iHealth"
(sqdbg) prof | mb 1 : total 519.10 us, avg 519.62 ns, peak   4.70 us(1), hits 1000
(sqdbg) prof | mb 2 : total  53.53 ms, avg 535.34 ns, peak  14.00 us(13015), hits 100000
(sqdbg) prof | mb 3 : total 525.01 ms, avg 525.02 ns, peak  19.70 us(167468), hits 1000000
---
(sqdbg) prof | tf 1 : total 684.60 us, avg 685.29 ns, peak   4.10 us(1), hits 1000
(sqdbg) prof | tf 2 : total  69.62 ms, avg 696.21 ns, peak  16.30 us(63146), hits 100000
(sqdbg) prof | tf 3 : total 696.02 ms, avg 696.02 ns, peak  32.50 us(617545), hits 1000000

// "m_Local.m_TonemapParams.m_flAutoExposureMin"
(sqdbg) prof | mb 1 : total 575.30 us, avg 575.88 ns, peak   5.80 us(1), hits 1000
(sqdbg) prof | mb 2 : total  56.62 ms, avg 566.22 ns, peak  10.70 us(85947), hits 100000
(sqdbg) prof | mb 3 : total 561.32 ms, avg 561.32 ns, peak  23.20 us(441406), hits 1000000
---
(sqdbg) prof | tf 1 : total   1.05 ms, avg   1.05 us, peak   4.20 us(1), hits 1000
(sqdbg) prof | tf 2 : total 105.56 ms, avg   1.06 us, peak  26.60 us(87595), hits 100000
(sqdbg) prof | tf 3 : total   1.06  s, avg   1.06 us, peak  24.30 us(479075), hits 1000000

Script_SpawnEntityGroupFromTable stack allocates unsanitised user input, but I digress, I'm not reviewing their code.

It's odd that SendToConsole doesn't have a whitelist, CSGO had it.

TF2 has a few debugoverlay functions individually exposed, mapbase uses debugoverlay singleton like in Source 2.

TF2 includes the worst function ever ScriptTraceLinePlayersIncluded, which was a function Valve added to CSGO in an operation in 2021 to use for turrets. They only had to add a mask parameter but they care not about reusability. Anyway, TF2 shouldn't have included this.

They commented out ConnectOutputs for the reason of regexp not save/restoring while they could have just put the object inside the function as a local.

ScriptDebug utility from L4D2 looks neat but I haven't used it before. They are also now obsolete with how convenient it is to use a real debugger, but they should be harmless apart from ScriptDebugHook - the hook conflicts with the debugger. This is an issue with Squirrel though.

I think having QAngle as a different type to Vector is a bad change. Extra type correctness would only be beneficial for new developers, and a hassle for anything else. It prevents script optimisations and requires extra allocations. I don't see any benefit to this.

TF2 doesn't have z33ky's stack allocation optimisation. I think it's worth keeping it.