[API Proposal]: System.Diagnostics.CodeAnalysis.StringSyntaxAttribute

Background and motivation

VS provides nice colorization of regular expressions when passed to known methods on Regex, e.g.

It also supports putting a comment before any string to mark the language being used in that string:

But there's currently no way to define an arbitrary API and annotate a string parameter as to what language is expected, e.g. you don't get this colorization today for RegexGenerator, because VS hasn't been taught about this API specifically:

While these examples are about regex, more generally this applies to arbitrary domain-specific languages. If VS adds colorization for JSON, for example, it'd be nice if every API that accepts JSON could be annotated appropriately.

API Proposal

namespace System.Diagnostics.CodeAnalysis
{
    [AttributeUsage(AttributeTargets.Parameter | AttributeTargets.Field | AttributeTargets.Property, AllowMultiple = false, Inherited = false)]
    public sealed class StringLanguageAttribute : Attribute
    {
        public StringLanguageAttribute(string language);
        public string Language { get; }
    }
}

API Usage

public sealed class RegexGeneratorAttribute : Attribute
{
    public RegexGeneratorAttribute([StringLanguage("regex")] string pattern);
}

Alternative Designs

No response

Risks

No response

@CyrusNajmabadi

cc: @CyrusNajmabadi, @jasonmalinowski, @terrajobst

Related to dotnet/roslyn#34640

@joperezr

cc: @joperezr

FWIW, i would love this. Note: we're also leaving design space open with Raw String Literals to allow for a direct way in the language to do it. Specifically:

var x = """regex
        \G(\w+\s?\w*),?
        """;

Embedded languages being a first class concept would be great. We would also like to do dotnet/roslyn#42634 so that 3rd parties could then implement the LS support for a particular embedded language (instead of Roslyn being responsible for it).

AttributeTargets.Parameter

Though I can't think of any concrete use cases off-hand, my gut reaction is that Field and Property seem like reasonable additions since this is meant to be more general than just Regex.

@CyrusNajmabadi

Though I can't think of any concrete use cases off-hand, my gut reaction is that Field and Property seem like reasonable additions since this is meant to be more general than just Regex.

Yeah, I had that initially, but it seemed like there would be various places where it'd be challenging to reflect that in a tool, e.g. passing such a field by ref would lose that information, and potentially ambiguities as to whether it should be applied, e.g. on a property, is that meant to impact assigning to the property and/or the code in the property returning a string. Combined with my not having scenarios in mind for it, I left it off. But, I'd leave this up to @CyrusNajmabadi and @jasonmalinowski and friends, if they expect it would be useful and feasible initially. It could always be added later.

so that 3rd parties could then implement the LS support for a particular embedded language

Cool. We have APIs in runtime that accept regexes, XML, and JSON; I expect we'd want to annotate many of those with this, e.g.

runtime/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs

Line 117 in 12a8819

    
           public static object? Deserialize(string json, Type returnType, JsonSerializerOptions? options = null)

We'll just need to standardize on a set of names for at least the languages we use, although maybe we would just snap to something like what GitHub uses for markdown language names. Thankfully, in general, the naming choice is obvious.

The string is Json (Upper case J). Lower case like above doesn’t work (at least in preview 1) and threw me off for a while.

runtime/src/libraries/System.Private.CoreLib/src/System/Diagnostics/CodeAnalysis/StringSyntaxAttribute.cs

Lines 40 to 47 in 785f818

    
           /// <summary>The syntax identifier for strings containing date and time format specifiers.</summary> 
        
           public const string DateTimeFormat = nameof(DateTimeFormat); 
        
           /// <summary>The syntax identifier for strings containing JavaScript Object Notation (JSON).</summary> 
        
           public const string Json = nameof(Json); 
        
           /// <summary>The syntax identifier for strings containing regular expressions.</summary> 
        
           public const string Regex = nameof(Regex);

@madelson

@madelson I'm not opposed to adding annotations for languages that Roslyn has no plans to support; this would allow other tools such as Rider to add support. However, I think we'll want at least one party that does something useful with it to ensure the annotations are sensible. For example, if you only ever want syntax highlighting, then sure, "SQL" might be good enough with a large dictionary of the various keywords used in SQL. However, if an IDE wants to do something more, such as code completion, then the string "SQL" might no longer be good enough. In the context of the BCL, "regex" basically means System.Text.RegularExpression, so that's in practice sufficiently unique. However, in .NET today there are various SQL dialects in use (MS SQL Server, Oracle, Postgress, etc). I can be convinced that this isn't a problem, but I'd like to hear this from an implementing party, not from a consumer because we'd likely make faulty assumptions.

did we add any other constants for well-known languages? The approved shape only had one for Regex.

We added Json, which is in the shown final shape as being fine to add when there's support, and we added DateTimeFormat, as you proposed and was discussed here #62505 (comment).

@stephentoub

@stephentoub in the Community Standup you mentioned keeping StringSyntaxAttribute internal if we implement it for codebases not targeting .NET 7.

For a multi-targeting public API (think nuget package) the best workaround I can think of without risking assembly conflicts is

public class Class2
{
    public void MyMethod([StringSyntax("Regex")] string pattern) { }
}

#if !NET7_0_OR_GREATER
internal sealed class StringSyntaxAttribute : Attribute { ... }
#endif

Is there a way to expose StringSyntaxAttribute to older TFMs without assembly conflicts or is this a technical limitation?

For a multi-targeting public API (think nuget package) the best workaround I can think of without risking assembly conflicts is

What's wrong with that approach? I don't think that's so much a workaround as it is the recommended course of action if you need to use the attribute downlevel.

Making it internal works beautifully for me in my multi targeted project

Sorry, thought I had tested this and thought the attribute had to be public to take effect.
It works brilliantly - this is awesome!

sorry for dumb question: I ask about SQL. If I know which dialect we are using in our project, is there any way to colorize SQL keywords? I mean parse SQL not by VS itself, but I already have a piece of code that can grab SQL from the code (via Roslyn) + can parse it to the tokens, so I have an information about tokens and theoretically can provide this info to VS visualization layer somehow. is it feasible now?

@terrajobst

@terrajobst I hear what you're saying about there being different dialects, but in many if not most cases SQL strings are being passed to a more generic SQL library (DbCommand.CommandText, EF, Dapper, etc) so even if the IDE had the ability to handle specific flavors of SQL in most cases it would probably have to fall back to lowest common denominator anyway.

Waiting to get buy-in from an implementer makes sense; just offering that as a consumer this would be valuable; definitely more so for my use-cases than JSON or date formats. I also suppose that there doesn't need to be an official constant here so long as the library authors and IDEs agree on a value to use for this.

For SQL, see this comment: #65634 (comment)

[...] even if the IDE had the ability to handle specific flavors of SQL in most cases it would probably have to fall back to lowest common denominator anyway.

I'm not sure what "lowest common denominator" means in SQL. Sure, there's a certain subset of statements which work everywhere, but it's really quite restricted. The moment you e.g. want to get only X rows, SQL Server has TOP, PG/SQLite have LIMIT/OFFSET, etc. IMHO for anything to be useful around SQL, there needs to be some setting somewhere that says what the dialect is, which the analyzer or IDE would pick up.

We are making this extensible at the Roslyn layer. But it will be up to a different party entirely to provide any level of SQL support.

I'm not sure what "lowest common denominator" means in SQL. Sure, there's a certain subset of statements which work everywhere, but it's really quite restricted.

I imagine that if VS/VSCode/Rider were to implement this then it would start with just the base keywords (SELECT, JOIN, WHERE, etc) and maybe some operators. I've worked with SQL in editors with this behavior and a little syntax highlighting goes a long way towards making the code easier to read.

IDEs can of course go beyond this by offering configurable dialect highlighting like JetBrains does.

I'm not sure what "lowest common denominator" means in SQL. Sure, there's a certain subset of statements which work everywhere, but it's really quite restricted.

I imagine that if VS/VSCode/Rider were to implement this then it would start with just the base keywords (SELECT, JOIN, WHERE, etc) and maybe some operators. I've worked with SQL in editors with this behavior and a little syntax highlighting goes a long way towards making the code easier to read.

IDEs can of course go beyond this by offering configurable dialect highlighting like JetBrains does.

We could indeed support only ANSI/ISO SQL (ISO/IEC 9075-1) by means of the identifier sql and then build on it in the long run with sql+mysql, sql+mssql/sql+tsql (maybe with version identifier? - sql+mysql8.0)...
That would be my approach now, anyway. Feel free to leave feedback.

build on it in the long run with sql+mysql, sql+mssql/sql+tsql...

As mentioned previously, likely most applications of this would be SQL dialect agnostic members like DbCommand.CommandText, parameters to various methods in Dapper, EF, etc. The value is in the enhanced readability of SQL blocks in an IDE as opposed to perfect parsing. Many text editors/IDEs/wikis/etc offer this sort of agnostic SQL syntax highlighting indicating that it is indeed useful despite the differences in SQL dialects.

Navigation Menu

Repository navigation

[API Proposal]: System.Diagnostics.CodeAnalysis.StringSyntaxAttribute #62505

Background and motivation

API Proposal

API Usage

Alternative Designs

Risks

53 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	/// <summary>The syntax identifier for strings containing date and time format specifiers.</summary>
	public const string DateTimeFormat = nameof(DateTimeFormat);

	/// <summary>The syntax identifier for strings containing JavaScript Object Notation (JSON).</summary>
	public const string Json = nameof(Json);

	/// <summary>The syntax identifier for strings containing regular expressions.</summary>
	public const string Regex = nameof(Regex);

[API Proposal]: System.Diagnostics.CodeAnalysis.StringSyntaxAttribute #62505

Description

Background and motivation

API Proposal

API Usage

Alternative Designs

Risks

Activity

stephentoub commented on Dec 7, 2021

stephentoub commented on Dec 7, 2021

CyrusNajmabadi commented on Dec 8, 2021

alexrp commented on Dec 8, 2021

stephentoub commented on Dec 8, 2021

stephentoub commented on Dec 8, 2021

53 remaining items

dotMorten commented on Feb 18, 2022

terrajobst commented on Feb 18, 2022

stephentoub commented on Feb 18, 2022

jnyrup commented on Feb 18, 2022

stephentoub commented on Feb 18, 2022

dotMorten commented on Feb 18, 2022

jnyrup commented on Feb 18, 2022

lsoft commented on Feb 19, 2022

madelson commented on Feb 19, 2022

roji commented on Mar 3, 2022

CyrusNajmabadi commented on Mar 4, 2022

madelson commented on Mar 4, 2022

deeprobin commented on Mar 4, 2022

madelson commented on Mar 5, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions