Respawn states “single line” of Apex Legends code lagged evasive audio bug

After the release of Season 16 of Pinnacle Legends, the developer Respawn found that a sneaky audio error had been caused by a single line of code.

Developers explained the problem in the Pinnacle subreddit, saying they found it when players complained that grenades sometimes caused damage without really exploding. During Season 16 playtesting, the issue was never encountered, and it was not possible to replicate the issue internally (at least not at first).

At first, the team suspected that the technique Pinnacle employs to send “stop”/”begin” orders for various impacts was at blame; they went on to explain that because the game’s servers reproduce elements (such as weapons, capabilities, and treasure), they would also produce noise and visual effects.

In the past, any results that numbered more than 128 on the server’s end would be discarded because of the restriction. The committee decided to send out a list to “any gamers who need the results for the specific server frame.”

From there, it was hypothesised that something was constantly overwhelming the engine with requests for new results. Do we have a systematic issue, or might it just be one person playing up? Countless changes are made to items, code, script, and levels with each seasonal update. That’s the equivalent of finding a needle in a haystack.

Although metrics are usually helpful at this stage in order to track down the origin of a problem, Respawn said that its telemetry data was useless since it “didn’t show any flags or problems in the system.”

Respawn continued, “This left us with a complex problem that we knew was affecting our neighbourhood but was tough to replicate despite comprehensive reports, had very few leads internally, and there were no metrics to show definitively that this limitation was being struck at all.”

After analysing the system’s many limitations, the team was able to simulate the problem in artificial settings, such as having a squad of 50 legends all fire their weapon at once. That provided hard evidence that FX will be abandoned, but only in “entirely impractical” test circumstances.

As the investigation progressed, Respawn said it kept a watch on additional reports exposing the issue and found that it seemed to occur more frequently during “top-level play.” down order to continue honing down on a certain niche, the team has published an updated server with new metrics for measuring the server’s influence on the network. While that update was sinking in, inspiration struck.

The problem was traced back to a single line of code. The brand-new season 16 weapon. The Nemesis has a particle effect that strengthens with use, but we can save money by not releasing the effects while it is not fully charged. If the weapon had no charge, Respawn said, “we would simply halt this particle impact when the weapon is upgraded.”

When the server acts as if it were the weapon’s owner, this line of code is executed. Each frame rendered on a client is accompanied by a set of player inputs, which the server must repeat. Since this piece of code was being run even when the weapon was holstered, it follows that every frame, any player with an uncharged Nemesis would generate a’stop particle”result’ on the server.

Because the issue was directly related to the frame rate of every client who had a Nemesis, Respawn said that it “immediately described” why the problem was more prevalent at higher levels of play. “14 customers with a Nemesis performing at 180FPS would suffice to trigger FX to start being dropped,” it said.

That also explains why the bug wasn’t discovered during internal testing: the builds used for testing may not have had enough holstered Nemesis in play, may have had a rarer connection with missing out on FX, or may simply not have had sufficient customers at that FPS.

The development team has said they would look into improving their screening techniques in the future, but have also said that “a minute of gamers playing Pinnacle is the equivalent of 10 testers playing the video game for a year,” which explains how rare flaws may be missed.

Check out the full explanation posted by the development team on Reddit for more details on how Respawn identified and fixed the issue.

.

Breaking the code

After examining the numerous constraints of the system, the group had the ability to recreate the issue in synthetic scenarios– such as by getting a team of 50 legends to all fire their weapon concurrently. That supplied concrete proof that FX would get dropped, however just throughout “entirely impractical” test cases.

As the examination advanced, Respawn stated it kept a close eye on brand-new reports highlighting the problem, and discovered that it appeared to happen more regularly throughout “top-level play.” With a fresh subset of video games to scan, the group released a server upgrade to include some metrics that would generate more information about its server impact networking, allowing the studio to continue narrowing its focus. As that upgrade was settling, the eureka minute lastly took place.

“A single line of code was recognized to be the source of the concern. Season 16’s brand-new weapon. The Nemesis has a particle result that increases when warming up, however when it’s uncharged we do not require to squander resources playing it as the results are hidden within the weapon. When the weapon gets upgraded we would merely stop this particle impact if the weapon had no charge,” stated Respawn.

“Every time the server mimics an input from the weapon’s owner, this specific line of code is performed. Gamers send out in their inputs for each single frame that is worked on their customer, and it’s the server’s task to replicate all of these inputs. This implies that each and every single gamer with an uncharged Nemesis would produce a ‘stop particle’ ‘result’ on the server every frame, and this line of code was being called even when the weapon was holstered.”

Respawn stated that “right away described” why the concern was happening more frequently at top-level play, due to the fact that it straight associated to the frame rate of each customer that had a Nemesis. “14 customers with a Nemesis performing at 180FPS would suffice to trigger FX to start being dropped,” it included.

That likewise describes why the bug wasn’t found throughout internal screening, due to the fact that the builds utilized for screening may not had actually had actually enough holstered Nemesis in play, might have had a rarer connection with missing out on FX, or may just have actually been missing out on sufficient customers at that FPS.

The dev group promised to take a look at enhancing its screening procedures in the future, however likewise mentioned that “a minute of gamers playing Pinnacle is the equivalent of 10 testers playing the video game for a year,” which discusses why uncommon bugs frequently slip through the internet.

For additional information on how Respawn determined and repaired the problem, take a look at the dev group’s complete explainer on Reddit.

Add to favorites