Support of Visual Studio 2019 in PVS-Studio, Part 1

DZone 's Guide to

Support of Visual Studio 2019 in PVS-Studio, Part 1

In this article, I will explain what problems we encountered when implementing support of the IDE and how we addressed them.

· Web Dev Zone ·
Free Resource

Support of Visual Studio 2019 in PVS-Studio affected a number of components: the plugin itself, the command-line analyzer, the cores of the C++ and C# analyzers, and a few utilities. In this article, I will explain what problems we encountered when implementing support of the IDE and how we addressed them.

Picture 2Before we start, I'd like to take a look back at the history of supporting the previous versions of Visual Studio in PVS-Studio so you better understand our vision of the task and solutions that we came up with in every single situation.

Since the first version of PVS-Studio that shipped with a plugin for Visual Studio (it was Visual Studio 2005 back then), supporting new versions of this IDE has been quite a trivial task for us, which basically came down to updating the plugin's project file and dependencies of Visual Studio's various API extensions. Every now and then we would have to add support for new features of C++, which the Visual C++ compiler was gradually learning to work with, but it generally wasn't a difficult task either and could be easily done right before a new Visual Studio release. Besides, PVS-Studio had only one analyzer back then — for C/C++.

Things changed when Visual Studio 2017 released. In addition to huge changes to many of the IDE's API extensions, we also encountered a problem with maintaining backward compatibility of the new C# analyzer added shortly before that (as well as of the new analyzer layer for C++ to work with MSBuild projects) with the new versions of MSBuild/Visual Studio.

Considering all of this, I strongly recommend that you see a related article about support of Visual Studio 2017, "Support of Visual Studio 2017 and Roslyn 2.0 in PVS-Studio: sometimes it's not that easy to use ready-made solutions as it may seem", before reading on. That article discusses the issues that we faced last time and the model of interaction between different components (such as PVS-Studio, MSBuild, and Roslyn). Knowing these details may help you to better understand the current article.

Tackling those problems ultimately led up to significant changes to the analyzer, and we were hoping that the new approaches applied then would help us support future versions of Visual Studio/MSBuild much easier and faster. This hope already started to prove realistic as the numerous updates of Visual Studio 2017 were released. Did the new approach help us in support of Visual Studio 2019? Read on to find out.

PVS-Studio Plugin for Visual Studio 2019

The start seemed to be promising. It didn't take us much effort to port the plugin to Visual Studio 2019 and have it launch and run well. But we already encountered two problems at once that could bring more trouble later.

The first had to do with the IVsSolutionWorkspaceService interface used to support of the Lightweight Solution Load mode (which, by the way, had been disabled in one of the earlier updates, back in Visual Studio 2017). It was decorated with the Deprecated attribute, which currently only triggered a warning at build time but was going to become a big problem in the future. This mode didn't last long... That was easy to fix — we simply stopped using this interface.

The second problem was the following message that we kept getting when loading Visual Studio with the plugin enabled: Visual Studio has detected one or more extensions that are at risk or not functioning in a feature VS update.

The logs of Visual Studio launches (the ActivityLog file) helped to clear it up:

Warning: Extension 'PVS-Studio' uses the 'synchronous auto-load' feature of Visual Studio. This feature will no longer be supported in a future Visual Studio 2019 update, at which point this extension will not work. Please contact the extension vendor to get an update.

What it meant for us was that we would have to switch from synchronous to asynchronous load mode. I hope you won't mind if I spare you the details of how we interact with Visual Studio's COM interfaces, and only briefly outline the changes.

There's an article by Microsoft about loading plugins asynchronously: "How to: Use AsyncPackage to load VSPackages in the background." It was, however, already clear that there were more changes to come.

One of the biggest changes was in the load mode, or rather initialization mode. In earlier versions, all the necessary initialization was done using two methods: Initialize our class inheriting from Package, and OnShellPropertyChange. The latter had to be added because when loading synchronously, Visual Studio itself might still be in the process of loading and initialization, and, therefore, some of the necessary actions were impossible to perform during the plugin's initialization. One way to fix this was to delay the execution of those actions until Visual Studio quits the 'zombie' state. It was this part of the logic that we singled out into the OnShellPropertyChange method with a check for the 'zombie' status.

The Initialize method of the abstract class AsyncPackage, which asynchronously loading plugins inherit from, is sealed, so initialization has to be done in the overridden method InitializeAsync, which is exactly what we did. The 'zombie' check logic had to be changed too because the status information was no longer available to our plugin. Besides, we still had to perform those actions that had to be done after plugin initialization. We solved that by utilizing the OnPackageLoaded method of the IVsPackageLoadEvents interface, which is where those delayed actions were performed.

Another problem resulting from asynchronous load was that the plugin's commands couldn't be used until after Visual Studio had loaded. Opening the analyzer log by double-clicking in the file manager (if you needed to open it from Visual Studio) resulted in launching the corresponding version of devenv.exe with a command for opening the log. The launch command looked something like this:

"C:\Program Files (x86)\Microsoft Visual Studio\
/command "PVSStudio.OpenAnalysisReport 

The "/command" flag is used here to run the command registered in Visual Studio. This approach didn't work anymore since commands were no longer available until after the plugin had loaded. The workaround that we came up with was to have the devenv.exe launch command parsed after the plugin has loaded and run the log open command if it's found in the launch command. Thus, discarding the idea of using the "appropriate" interface to work with commands allowed us to keep the necessary functionality, with delayed opening of the log after the plugin has completely loaded.

Phew, looks like we made it at last; the plugin loads and opens as expected, without any warnings.

And here's when things go wrong. Paul (Hi Paul!) installs the plugin on his computer and asks why we still haven't switched to asynchronous load.

To say that we were shocked would be an understatement. That couldn't be! But it's real: here's the new version of the plugin, and here's a message saying that the package is loading synchronously. Alexander (Hi Alexander!) and I try the same version on our respective computers — it works fine. How's that possible? Then it occurs to us to check the versions of the PVS-Studio libraries loaded in Visual Studio - and we find that these are the libraries for Visual Studio 2017, whereas the VSIX package contains the new versions, i.e. for Visual Studio 2019.

After tinkering with VSIXInstaller for a while, we managed to find out that the problem had to do with the packages cache. This theory was also supported by the fact that restricting access to the cached package (C:\ProgramData\Microsoft\VisualStudio\Packages) caused VSIXInstaller to output an error message in the log. Curiously enough, when the error didn't occur, the information about installing cached packages didn't appear.

Side note: While studying the behavior of VSIXInstaller and accompanying libraries, I thought how cool it is that Roslyn and MSBuild are open-source, which allows you to conveniently read and debug their code and trace its work logic.

So, this is what happened: when installing the plugin, VSIXInstaller saw that the corresponding package was already cached (it was actually the .vsix package for Visual Studio 2017) and installed that package instead of the new one. Why it ignored the restrictions/requirements defined in the .vsixmanifest file (which, among other things, restricted installation of extensions to a specific version of Visual Studio) is a question yet to be answered. As a result, the plugin designed for Visual Studio 2017 got installed on Visual Studio 2019 — despite the restrictions specified in the .vsixmanifest file.

Worst of all, that installation broke the dependencies graph of Visual Studio, and although the IDE seemed to be running well, things were actually terrible. You couldn't install or delete extensions, update, etc. The "restore" process was painful too as we had to delete the extension (i.e. the files comprising it) manually and — also manually — edit the configuration files storing the information about the installed package. In other words, it wasn't fun at all.

To fix that and to make sure we didn't run into any situations like that in the future, we decided to make our own GUID for the new package to have the packages for Visual Studio 2017 and Visual Studio 2019 securely isolated from each other (the older packages were fine; they had always used a shared GUID).

Since we started talking about unpleasant surprises, here's another: after updating to Preview 2, the PVS-Studio menu "moved" to the "Extensions" tab. Not a big deal, but it made accessing the plugin's functionality less convenient. This behavior persisted through the next Visual Studio 2019 versions, including the release. I have found mentions of this "feature" in neither the documentation nor in the blog.

Okay, now things looked fine and we seemed to have finished with the Visual Studio 2019 support at last. This proved wrong the next day after releasing PVS-Studio 7.02. It was the asynchronous load mode again. When opening the analysis results window (or starting the analysis), the analyzer window would appear "empty" to the user — no buttons, no grid, nothing at all.

This problem, in fact, occurred every now and then during the analysis. But it affected only one computer and didn't show up until Visual Studio updated to one of the first iterations of 'Preview.' We suspected that something had got broken during installation or update. The problem, however, disappeared some time later and wouldn't occur even on that particular computer, so we thought it "got fixed on its own." But no — we just were lucky. Or unlucky, for that matter.

As we discovered, it was the order in which the IDE window itself (the class derived from ToolWindowPane) and its contents (our control with the grid and buttons) were initialized. Under certain conditions, the control would be initialized before the pane and even though things ran well and the FindToolWindowAsync method (creating the window when it's accessed for the first time) did its job well, the control remained invisible. We fixed that by adding lazy initialization for our control to the pane-filling code.

Support of C# 8.0

There's one great advantage about using Roslyn as a basis for the analyzer: you don't have to add support for new language constructs manually — it's done automatically through the Microsoft.CodeAnalysis libraries, and we just make use of the ready-made solutions. It means new syntax is supported by simply updating the libraries.

As for the analysis itself, we had to tweak things on our own, of course — in particular, handle new language constructions. Sure, we had the new syntax tree generated automatically by simply updating Roslyn, but we still had to teach the analyzer how exactly to interpret and process new or modified syntax tree nodes.

The nullable reference types are perhaps the most widely discussed new feature of C# 8. I won't be talking about them now because a topic that big is worth a separate article (which is currently being written). For now, we have settled on ignoring nullable annotations in our dataflow mechanism (that is, we understand, parse, and skip them). The idea is that a variable, even of a non-nullable reference type, can still be pretty easily (or accidentally) assigned the value null, ending up with an NRE when attempting to dereference it. Our analyzer can spot such errors and report a potential null dereference (if it finds such an assignment in the code, of course) even if the variable is of type non-nullable reference.

Using nullable reference types and associated syntax enables you to write pretty interesting code. We nicknamed it "emotional syntax." This snippet is perfectly compilable:


By the way, my experiments led me to discovering a couple of tricks that you can use to "crash" Visual Studio using the new syntax. They are based on the fact that you are allowed to write as many '!' characters as you like. It means you could write not only code like this:

object temp = null!

but also like this:

object temp = null!!!;

And, pushing it even further, you could write crazy things like this:

object temp = null!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

This code is compilable, but if you try to view the syntax tree in Syntax Visualizer from .NET Compiler Platform SDK, Visual Studio will crash.

The failure report can be pulled out from Event Viewer:

Faulting application name: devenv.exe,
version: 16.0.28803.352, time stamp: 0x5cc37012
Faulting module name: WindowsBase.ni.dll,
version: 4.8.3745.0, time stamp: 0x5c5bab63
Exception code: 0xc00000fd
Fault offset: 0x000c9af4
Faulting process id: 0x3274
Faulting application start time: 0x01d5095e7259362e
Faulting application path: C:\Program Files (x86)\
Microsoft Visual Studio\2019\Community\Common7\IDE\devenv.exe
Faulting module path: C:\WINDOWS\assembly\NativeImages_v4.0.30319_32\
Report Id: 66d41eb2-c658-486d-b417-02961d9c3e4f
Faulting package full name: 
Faulting package-relative application ID:

If you go even crazier and add several times more exclamation points, Visual Studio will start crashing all by itself, without any help from Syntax Visualizer. The Microsoft.CodeAnalysis libraries and the csc.exe compiler can't cope with such code either.

These examples are contrived, of course, but I found that trick funny.

That's all for Part 1! Tune back in for Part 2, where we'll talk about toolsets, changes to the mechanism of evaluating C++ projects, and changes to the mechanism of evaluating C# .NET Core projects.

.net, .net core, c#, roslyn, static analysis, visual studio, web dev

Published at DZone with permission of Sergey Vasiliev . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}