When you build an application with MsBuild and/or Visual Studio you work with at least 2 types of files:
•The solution file (with the .sln extention)
•One or more project files. XSharp projects have the .xsproj extension. CSharp projects have the .csproj extension, Visual Basic projects the .vbproj extension.
The solution file (.sln)
The solution file is a text file with a list of project files and other information. Each project entry looks like this:
Project("<language guid>") = "<ProjectName>", "<Path and filename of the project file>", "<project guid>"
EndProject
The <language guid> is always "{AA6C8D78-22FF-423A-9C7C-5F2393824E04}" for X# projects. This tells Visual Studio which project system to use to open the project file.
The <project guid> is a generated and should match the project GUID that is defined inside the .xsproj file. These guids are also used in other sections of the .sln file.
Other language guids that you may see are {FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} and {9A19103F-16F7-4668-BE54-9A1E7A4F7556} for C# and {2150E333-8FDC-42A3-9474-1A3956D46DE8} for subfolders in your solution. There are many more guids of course.
Solution files also contain sections that describe the various configurations that are available for the solution (such as "Debug" and "Release") and a section that maps solution configurations to project configurations and sometimes also a section that indicates how the source code control bindings for each of the projects are.
For the actual build process of your X# apps we can ignore the solution file for now. Solution files are "language agnostic". The building is done based on information in the project file.
The XSharp project file (.xsproj)
The project file contains all the instructions that are needed to build a X# project with MsBuild. The file is a Text file and contains XML contents in a specific format that MsBuild understands.
The file contains all the settings that you can set from the project properties dialogs in Visual Studio and also a list of the items (prg files, resx files, rc files etc) in the project.
The file uses some common information that is installed in a MsBuild subfolder inside the XSharp installation folder that belongs to the Visual Studio version that you are using.
The most important pieces in the file are for now:
Item |
Description |
<Import Project="$(MSBuildExtensionsPath)\XSharp\XSharp.Default.props" /> |
This imports default settings for XSharp from the XSharp folder inside the current MsBuild folder. This file contains several default values for XSharp and also imports default values from a common file delivered by Microsoft (Microsoft.Common.props) |
Several <PropertyGroup> sections. |
These sections contain values for the several options that you can find on the Project Properties dialog in Visual Studio. Some values are for all configurations, some values are configuration specific. |
One or more <ItemGroup> sections with <Reference> items |
The <Reference> items describe so called Assembly references that your project has. Usually you will find something like <Reference Include="System" /> in there. Reference Items may also contain more information such as a version number etc. These references will be converted to -reference command line options for the compiler |
One or more <ItemGroup> sections with <ProjectReference> items |
The <ProjectReference> items describe so called Project References to other projects inside the same solution. MsBuild will determine the build order inside the solution based on the various project references and will try to build the referenced projects first before the projects referencing them. MSBuild will include a reference to the file produced by the project reference when building the command line for the compiler. |
One or more <ItemGroup> sectins with <COMReference> items |
The <COMReference> items describe references to COM components. This may be automation servers (such as Word or Excel for example) or ActiveX components (like the Shell Explorer that we use in the email example). Automation Servers will have a single COMReference with a <wrappertool> child node of type "tlbImp". ActiveX controls will have 2 COMReferences. One with the wrappertool set to "tlbimp" and another with the wrappertool set to "aximp". See the section below on how this is processed by MSBuild. |
One or more <ItemGroup> sections with <Compile> items |
The <Compile> items describe the source code items for the X# compiler. The template for the Console application has 2 of these items: The <Compile> items may have an optional <SubType>child node with the value "Code", "Form" or "UserControl". This subtype is ignored by the build process but used by Visual Studio to determine the icon that is shown before the item in the tree and to determine which editor to open when the item is double clicked. "Code" opens the source code editor by default. The other 2 types open the Windows Forms editor.
|
One or more <ItemGroup> sections with <VOBinary>, <NativeResource>,<EmbeddedResource> and other types of items |
<NativeResource> items are handled specially by the X# build process. These are combined together in an unmanaged resource. See below |
<Import Project="$(MSBuildExtensionsPath)\XSharp\XSharp.targets" /> |
This file tells MSBuild how to handle the <Compile> and <NativeResource> items in the project file and also (indirectly) imports a file Microsoft.Common.targets that tells MSBuild how to handle XAML files and how to compile <EmbeddedResources>. |
How does MSBuild locate referenced assemblies
When locating the referenced assemblies needed for compiling your project it looks at the following:
1.When the reference node has a "hintpath" then it tries to locate the file through this path. That could look like <HintPath>..\\SDK_Defines.dll</HintPath>
2.When the reference node is a "normal" .Net framework assembly, it looks at the folder on your file system that matches the framework version. For example when the framework version of your project is 4.6 (there will be a node <TargetFrameworkVersion>v4.6</TargetFrameworkVersion> then it will look for System.DLL in the folder c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6
3.When the reference node is not a standard .Net framework assembly and the 3rd party vendor has registered a folder in a specific location then MsBuild will use that location. X# registers a folder in the HKLM\Software\Microsoft\.NETFramework\v4.0.30319\AssemblyFoldersEx\XSharp location. File registered in a location like this are also shown in the "Add References" dialogs in Visual Studio.
4.Finally (during building) MSBuild will look in the Global Assembly Cache (GAC).
Please note that there is a difference between Compiling and Running. During the compilation the files in the reference folders have preference over files in the GAC. When running the app the GAC is used and/or the local folder or path. Files in the reference folders are NEVER used at runtime. This would also not be possible since these files have no executable code in them.
The idea behind this division is that you can have a newer Framework version installed (for example 4.8) then you would like to compile against (for example 4.6). The reference assembly in the 4.6 folder only contains the subset of the api that was available for .Net Framework 4.6. So you will not be able to (accidentally) use methods or types that were added after framework 4.6, even though these methods and/or types are installed in the GAC.
How does MSBuild locate project references
When MSBuild delects a project reference then it tries to build that project first. When the project is successfully compiled then the output assembly from that project is included as "normal" reference to the X# compiler.
How are COM references handled
COM references require special processing. MSBuild uses 2 command line tools to extract the type libraries from these COM references and produces .Net assemblies (so called Interop assemblies) that describe the COM references. There are 2 tools involved:
tlbimp.exe for automation servers
aximp.exe for ActiveX controls
In our email example we are using the Shell.Explorer Active X. These 2 tools produce the files Interop.SHDocVw.dll and AxInterop.SHDocVw.dll . The AxInterop file describes the Windows Forms control and the Interop file the automation interface. In our Excel example we are referencing a "precompiled" assembly for Office and therefore we are not generating a new interop assembly but we are linking to a so called "Primary Interop Assembly (PIA)", with the name Microsoft.Office.Interop.Excel.dll .
If you include a COM component but you are not actually creating the COM objects but only consuming them then you can also set the "EmbedInteropTypes" option to true. When you do that then the X# compiler will copy the relevant information from the interop assembly and include that exe or dll, so you do not have to distribute the interop.dll with your application. In the Excel example that will not work because we are creating an excel application. The compiler will complain then "error XS1752: Interop type 'Microsoft.Office.Interop.Excel.ApplicationClass' cannot be embedded. Use the applicable interface instead."
The resulting interop assemblies are produced before the compiler is called and are passed to the compiler as "normal" assembly references.
How does MSBuild call the Native Resource compiler
When your application contains Native resources then we must compile these native resources before the X# compiler can be used, since the result of the resource compilation must be included in the final exe/dll file. Of course MSBuild does not "know" about X#, so we have to tell it what to do. The instructions for this are stored in the XSharp.Targets file.
This file contains the following instructions:
<UsingTask TaskName="NativeResourceCompiler" AssemblyFile="$(MSBuildThisFileDirectory)XSharp.Build.dll" />
<NativeResourceCompiler> .... </NativeResourceCompiler>
The first entry tells the compiler that there is a special DLL in the XSharp folder with the name XSharp.Build.dll. This DLL contains a type NativeResourceCompile, which is a subtype of Microsoft.Build.Utilities.ToolTask.
The second entry tells MSBuild how to pass information to this task to build the native resources.
This includes a list of all items from the project file with the itemtype <NativeResource>.
The task will then try to find the native resource compiler. To do that it looks in the registry in the following key:
- When running in 64 bit mode: "HKEY_LOCAL_MACHINE\Software\WOW6432Node\XSharpBV\XSharp"
- When running in 32 bit mode: "HKEY_LOCAL_MACHINE\Software\XSharpBV\XSharp"
Inside this key it looks for the (string) value XSharpPath which is set by the installer at compile time.
When it can't find that path it defaults to "C:\Program Files (x86)\XSharp"
The task will then look for the rc.exe program in the Bin subfolder below that folder.
When the tool is found then this task checks for the date/time stamps of the various .rc files and compares these with the date/time stamp of the output file (NativeResources.res ) in the "intermediate" folder. If the output file is older or does not exist then a command line for rc.exe is constructed and the compiler is called.
For this call we create a unique temporary rsp file in your temp folder. We are also saving the last version of this file in the file "LastXSharpNativeResourceResponseFile.Rsp".
If you want to see which information was passed to the native resource compiler you can look for this file in your temp folder.
The resulting NativeResources.res will be passed to the X# compiler later to be included in the exe/dll. For this we use the /win32res command line option of xsc.exe.
How are managed resources compiled
The compilation process for managed resources is mostly managed by MSBuild itself. It already knows how to handle these.
We do declare a task
<UsingTask TaskName="CreateXSharpManifestResourceName" AssemblyFile="$(MSBuildThisFileDirectory)XSharp.Build.dll"/>
<CreateXSharpManifestResourceName> ... </CreateXSharpManifestResourceName>
This task is also located in the same XSharp.Build.DLL and is used to help MSBuild to detect the right namespace for the generated resources.
The result of the managed resource compilation is that .resx files are compiled to one or more .resources file. These .resources files are then later passed to the compiler with the /resources command line option of xsc.exe.
Processing of XAML files
If you are creating a project that contains WPF windows or controls then an extra step is needed to produce the exe/dll.
In this step MSBuild produces so called .baml files and calls a code generator to generate source code for each XAML file.
For the WPF template 2 source files are produced:
•WPFWindow1.g.prg
•App.g.prg
These source files are automatically added to the command line for the X# compiler.
These source files contain a class declaration with a InitializeComponent() method that sets up the controls in your window. If you have named your controls then for each control with a name there will also be a field in the class and the generated Connect() method will set these fields to the control generated by the framework when the form is loaded.
App.g.prg also contains a class and a function Start() that is responsible for starting up your application.
Note: This source code is generated by a tool that we have registered in c:\Windows\Microsoft.NET\Framework\v4.0.30319\Config\machine.config .
<system.codedom>
<compilers>
<compiler language="XSharp" extension=".prg" type="XSharp.CodeDom.XSharpCodeDomProvider,XSharpCodeDomProvider, Version=2.1.0.0, Culture=neutral,
PublicKeyToken=ed555a0467764586, ProcessorArchitecture=MSIL" />
</compilers>
The XSharpCodeDomProvider.dll assembly is registered in the GAC and it contains a XSharpCodeGenerator type that is responsible for the code generation.
Note: this tool uses the keyword case setting that was specified in your Visual Studio options for the X# text editor.
How does MSBuild call the X# compiler
When MSBuild has successfully handled all external references and has created the "code behind" for XAML files compiled the native and managed resources then it calls the X# compiler.
Similar to how the native resource compiler is called the XSharp.Targets file also has instructions on how to call the compiler:
<UsingTask TaskName="XSharp.Build.Xsc" AssemblyFile="$(MSBuildThisFileDirectory)XSharp.Build.dll"/>
<Xsc> ..... </Xsc>
Again this describes a class in the XSharp.Build.DLL and the <Xsc> entry describes the properties of this type that need to be set.
The Xsc task looks for the xsc.exe just like how the native resource compiler does this:
- It looks for the installation location in the registry
- It defaults to the "C:\Program Files (x86)\XSharp" folder
There is one difference:
- It also looks for an environment variable "XSHARPDEV". When this environment variable exists it assumes that this is an alternate location where it can find the xsc.exe. We are using this internally so we can compiler with a newer version of the compiler than the one that is installed inside C:\Program Files (x86)\XSharp. You may use if you want to work with more than one version of the compiler on your machine.
When we can find the xsc.exe compiler then we construct the command line to the compiler. We are creating a unique temporaty RSP file in the temp folder, just like we do for the native resource compiler. We are also saving the last version of this file to the "LastXSharpResponseFile.Rsp" file in that folder.
If you have enabled the "Shared" compiler on the Build page in your project properties (this defaults to true) then we add the command line option /shared. This will tell xsc.exe to run XSCompiler.exe and pass the command line to that tool. XSCompiler.exe will continue to run in memory even after the compilation is finished and will cache type information from referenced assemblies. As a result a second compilation of the same project will usually be much faster, since all the relevant type information is already cached. Of course the compiler is smart enough to detect when a referenced DLL was changed (the reference could be generated from a referenced project) and will then reload the type information from that reference. Normally you will only see one copy of XSCompiler.exe running memory. You may see multiple copies of xsc.exe running in memory when MSBuild detects that 2 projects in the same solution are "independent" and can be compiled simultaneously.
The only situation where you might see 2 copies of XSCompiler.exe running in memory is when projects are compiled with difference settings for case sensitivity (the /cs command line option). One of the 2 copies will then have a case sensitive type cache and the other a case insensitive type cache.
Debugging MSBuild
If you want to see what MSBuild imports when compiling your xsproj file you can call MSBuild with a special commandline option. To do so open a visual studio developer command prompt and type the following:
msbuild -preprocess <yourproject.xsproj> > preprocessed.proj
The resulting preprocess.proj file will be an XML file that contains all imported instructions. You can open this inside Visual Studio. You may want to Format the document to make it a bit more readable.
You should see that all "<Import project" nodes are now converted to comments and the contents of these imported files is inserted into the preprocessed output.
Some imports had a condition that was not met and these are just in the file as comments.
The generated file is HUGE (the WPF template produces a file of over 8700 lines and some of these lines are thousands of characters wide. Almost all of the first 8600 lines of this preprocessed file are all imported.
Somewhere in this file you will see that MSBuild.
Please note that you are NOT able to build the output file. It just serves to see what MSBuild imports to create your project.
If you want to see how msbuild resolves the various references you should call msbuild from the command line and add the command line option to show detailed info. The /target:rebuild on the next line makes sure that everything is rebuilt. If you are compiling a project with native resources, managed resources or xaml files you should also see the logging of the tools that process this.
msbuild -verbosity:detailed <yourproject.xsproj> /target:rebuild >buildlog.txt
