linq - EF Core 2.1 to EF Core 8.NET 8 migration: Why does this EF Core InMemory query generate a 'Nullable object must h

After upgrading my .NET 5.0 project to .NET 8.0, the LINQ query now generates an exception

System.InvalidOperationException: Nullable object must have a value

The code below executes successfully in .NET 5.0 (with EF 2.1.1) but fails after upgrading to .NET 8.0 (with EF Core 8.0.12).

My DbContext and model are configured as shown here:

namespace Example
{
    public class Competitor
    {
        public string ID { get; set; }
    }

    public class Result
    {
        public string ID { get; set; }
        public string WinnerID { get; set; }
    }

    public class MyDbContext : DbContext
    {
        public MyDbContext()
            : base(new DbContextOptionsBuilder<MyDbContext>().UseInMemoryDatabase("MyDB").Options)
        {
        }

        public DbSet<Competitor> Competitors { get; set; }
        public DbSet<Result> Results { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<Competitor>()
                .HasKey(c => c.ID);

            modelBuilder.Entity<Result>()
                .HasKey(r => r.ID);
        }
    }
}

And here is the query that fails:

namespace Example
{
    public class UnitTest1
    {
        [Fact]
        public void TestOuterJoinToGroup()
        {
            using var context = new MyDbContext();

            context.Competitors.AddRange(new Competitor() { ID = "1" }, new Competitor() { ID = "2" });
            context.Results.AddRange(new Result() { ID = "1", WinnerID = "1" }, new Result() { ID = "2", WinnerID = "1" });
            context.SaveChanges();

            var individualSummary = context.Results.GroupBy(
                ic => ic.WinnerID,
                (WinnerID, ic) => new
                {
                    CompetitorID = WinnerID,
                    Wins = ic.Count()
                });

            var competitorResults = from c in context.Competitors
                                    join r in individualSummary on c.ID equals r.CompetitorID into groupJoin
                                    from outerJoin in groupJoin.DefaultIfEmpty()
                                    select new
                                    {
                                        Id = c.ID,
                                        Wins = outerJoin == null ? 0 : outerJoin.Wins,
                                    };
            var arrCompetitorResults = competitorResults.ToList();
        }
    }
}

What is the cause of the exception and what do I need to change to make the query execute successfully?

The full exception is:

System.InvalidOperationException: Nullable object must have a value.
   at lambda_method148(Closure, ValueBuffer)
   at System.Linq.Utilities.<>c__DisplayClass2_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Utilities.<>c__DisplayClass2_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNextHelper()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Example.UnitTest1.TestOuterJoinToGroup() in D:\source\TestProject1\TestProject1\UnitTest1.cs:line 73

After upgrading my .NET 5.0 project to .NET 8.0, the LINQ query now generates an exception

System.InvalidOperationException: Nullable object must have a value

The code below executes successfully in .NET 5.0 (with EF 2.1.1) but fails after upgrading to .NET 8.0 (with EF Core 8.0.12).

My DbContext and model are configured as shown here:

namespace Example
{
    public class Competitor
    {
        public string ID { get; set; }
    }

    public class Result
    {
        public string ID { get; set; }
        public string WinnerID { get; set; }
    }

    public class MyDbContext : DbContext
    {
        public MyDbContext()
            : base(new DbContextOptionsBuilder<MyDbContext>().UseInMemoryDatabase("MyDB").Options)
        {
        }

        public DbSet<Competitor> Competitors { get; set; }
        public DbSet<Result> Results { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<Competitor>()
                .HasKey(c => c.ID);

            modelBuilder.Entity<Result>()
                .HasKey(r => r.ID);
        }
    }
}

And here is the query that fails:

namespace Example
{
    public class UnitTest1
    {
        [Fact]
        public void TestOuterJoinToGroup()
        {
            using var context = new MyDbContext();

            context.Competitors.AddRange(new Competitor() { ID = "1" }, new Competitor() { ID = "2" });
            context.Results.AddRange(new Result() { ID = "1", WinnerID = "1" }, new Result() { ID = "2", WinnerID = "1" });
            context.SaveChanges();

            var individualSummary = context.Results.GroupBy(
                ic => ic.WinnerID,
                (WinnerID, ic) => new
                {
                    CompetitorID = WinnerID,
                    Wins = ic.Count()
                });

            var competitorResults = from c in context.Competitors
                                    join r in individualSummary on c.ID equals r.CompetitorID into groupJoin
                                    from outerJoin in groupJoin.DefaultIfEmpty()
                                    select new
                                    {
                                        Id = c.ID,
                                        Wins = outerJoin == null ? 0 : outerJoin.Wins,
                                    };
            var arrCompetitorResults = competitorResults.ToList();
        }
    }
}

What is the cause of the exception and what do I need to change to make the query execute successfully?

The full exception is:

System.InvalidOperationException: Nullable object must have a value.
   at lambda_method148(Closure, ValueBuffer)
   at System.Linq.Utilities.<>c__DisplayClass2_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Utilities.<>c__DisplayClass2_0`3.<CombineSelectors>b__0(TSource x)
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNextHelper()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Example.UnitTest1.TestOuterJoinToGroup() in D:\source\TestProject1\TestProject1\UnitTest1.cs:line 73

• linq
• entity-framework-core
• nullable
• ef-core-2.1
• ef-core-8.0

Share Follow edited Jan 31 at 10:06 Gert Arnold 109k3636 gold badges214214 silver badges313313 bronze badges asked Jan 30 at 8:27 colinsl77colinsl77 322 bronze badges 11

• Side note: in tests you should use separate context instances for setting up data and the "act" stage of the test. In real life, you also query existing data with a clean context. As for the issue, try Wins = (int?)outerJoin.Wins, although I can't tell why it fails now. – Gert Arnold Commented Jan 30 at 8:51
• The In-memory provider is essentially just a Dictionary, there's no SQL generation. The operators are applied directly to the objects and nuIls can result in NREs. If you want to test complex queries use SQLite in in-memory mode – Panagiotis Kanavos Commented Jan 30 at 9:04
• Besides, why join at all instead of using navigation properties, eg Result.Competitor or Result.Competitors, Competitor.Results, Competitor.Wins ? That's how EF is meant to work, and that works just fine with the in-memory provider. EF deals with entities, not tables. It's the provider's job to generate JOINs when needed, not the developer's – Panagiotis Kanavos Commented Jan 30 at 9:07
• In this case for example, it looks like you're trying to list all competitors and whether they won or not, context.Competitors.Select(c=>new{c.Id, Wins=c.Wins.Count()}).ToList(). If you don't want that count context.Competitors.Select(c=>new{c.Id, HasWin=c.Wins.Any()}).ToList() – Panagiotis Kanavos Commented Jan 30 at 9:12
• (with EF 2.1.1) then your query probably never worked and would crash even in EF Core 3. EF Core 2 had a lot of missing features so it silently loaded the objects in memory and performed client-side evaluation. In EF 2.1 you already got warnings for such queries. It was in EF Core 3, not 8, that the missing features were fully implemented and client-side evaluation was disabled, resulting in runtime errors – Panagiotis Kanavos Commented Jan 30 at 9:14

 |  Show 6 more comments

1 Answer 1

Sorted by: Reset to default 0

This is because until EF-core 2, EF auto switched to client-side evaluation when it encountered parts in a LINQ query that couldn't be translated into SQL. In your case, EFC 2.1.1, this apparently lead to client-side evaluation of the part outerJoin == null ? 0 : outerJoin.Wins. I'm not really sure, because in EFC 2.2.6 (the earliest version I can test easily) I get the same exception.

As of EF core 3, EF only supports client-side evaluation in the final Select but has also become smarter in evaluating the LINQ query before translation. Now it concludes that outerJoin.Wins is an int (not nullable) and it concludes that, technically, the outerJoin == null condition is always false and doesn't need translation. However, the data still gives rise to results where outerJoin.Wins is null.

This is an infamous glitch that can be hard to spot in more complex queries, but the solution is always to help EF by telling it that the result is nullable. In your query, this is done by the following construct:

var competitorResults = from c in context.Competitors
                        join r in individualSummary on c.ID equals r.CompetitorID into groupJoin
                        from outerJoin in groupJoin.DefaultIfEmpty()
                        select new
                        {
                            Id = c.ID,
                            Wins = (int?)outerJoin.Wins ?? 0,
                        };

Here, (int?)outerJoin.Wins tells EF that outerJoin.Wins can be null and ?? 0 tells it to convert null values to 0. In the SQL query this appears as COALESCE([p0].[Wins], 0) AS [Wins].