.NET open source diversity

[duracellko]

Question

I know that .NET is far from JavaScript world, where NPM has more packages than space shuttle has parts. Would you wish .NET community would go that way? More packages, more projects, diversity is everything.
Or would you redirect new enthusiastic contributors to rather explore and help existing solutions before starting anything new? If yes, how would you support that?

Background / More Information

When .NET Standard was about to come I said:
Java: Build once, debug everywhere.
JavaScript: One language to rule them all.
.NET: One library to rule them all.

When .NET started it was mostly compared to Java. And in my opinion one of big differences was amount of libraries/frameworks. In .NET there was single framework for desktop applications and single framework for web applications. That's what I liked about .NET.
While in Java there were several frameworks for desktop apps and several for web. And this pretty much defined open source world. And it got even more extreme in JavaScript world.

[jguadagno]

When .NET started, it started within the corporate enterprises, everything was under lock and key. In the past few years, with the advent of GitHub, NuGet, and some other things that have changed. Companies and developers are contributing to and using this vast amount of packages that are available. While it's still not at the level of NPM yet, I think we'll get there soon. I really believe this would continue to grow naturally as Microsoft and other organizations continue to Open Source and share there stuff.

[samidip]

@duracellko Great question. I think what the JS community has done with NPM packages is venerable, but .NET ecosystem may not exactly need to follow suit. .NET today powers apps across a wide variety of platforms and code reusability is the key. As you mention, enthusiasts should explore existing solutions first and make libraries portable, when possible. We should invest more energy in trying to make older code .NET Standard compliant and push the envelope with new cross-platform libraries. While truly universal UI layer may be difficult, modern .NET paves way for easy sharing of libraries and this is where open source contributions can really contribute to ecosystem richness. Cheers to that :)

[SeanKilleen]

Thanks for the question, @duracellko! Tagging @dotnet-foundation/candidates so they're aware aware this question has been asked.

[schneidenbach]

Very good question! This related directly back to ecosystem culture. Let's deconstruct your NPM comparison a bit. JS' culture of "move fast and break things" works well for the nascent JavaScript world, where they had to figure out a whole host of things that .NET solved a while ago (package management, build process, etc.) Many framework were popular at first, then quickly died as convergence took place (example: Bower -> NPM, Gulp/Grunt -> NPM/Webpack).

The distributed nature of NPM comes with tradeoffs that I think doesn't necessarily work for the .NET ecosystem. I'm slightly off topic, so now I think I'll answer your question 🙂

Thinking as a Foundation board member, I think we'd want to think about what people really want. Going back to your other question - for those want to roll up their sleeves and contribute to open source, the Foundation could assist with connecting the people who want help to the people that need help.

If there was an unmet need that a new open source contributor wanted to solve, I think another Foundation initiative could be connecting nascent contributors with people who could coach them.

Put simply:

New open source contributor/new project -> coaching
New open source contributor/existing project -> making opportunities to contribute more visible

I'd start by looking at what other similar foundations (GNOME?) have done to encourage this in the past - what worked for them, what didn't - and source more ideas from the community.

[jskeet]

I'd certainly say that the NuGet ecosystem has been greatly improved in the last few years as a result of .NET Core, both in terms of tooling and frameworks:

• It's easier to create packages
• It's easier to consume packages (bye bye, HintPath!)
• It's easier to understand what you are consuming
• It's easier to build packages and applications that are cross-platform (I don't miss PCL profile numbers one bit)

I do think we face one big technical hurdle in terms of NuGet packages that isn't being addressed: versioning. We're generally happy saying we support semantic versioning, but neither the language nor the runtime is really geared up for this. The diamond dependency problem is very real, and sooner or later it's going to bite us. This is something I've been concerned about for a long time, and to my shame I haven't written up the ideas I've had to potentially address it - maybe one day. But it's an area where I think the .NET Foundation should be leading the way in advocating for technical solutions to technical issues that will particularly hurt (or limit) the open source community.

Having said all that, I'd like to address the last part of your question:

Or would you redirect new enthusiastic contributors to rather explore and help existing solutions before starting anything new? If yes, how would you support that?

While I'd never particularly want to dissuade anyone from building anything new, I think it's always worth exploring what's available too. Then if you still want to start something new, at least you're doing so with your eyes open. (If you want to build another DI system, test framework or mocking framework, I'd definitely ask if you're really bringing anything new to the table.)

I don't know whether it's particularly within the .NET Foundation's remit or something that might work better within Microsoft's documentation, or even as part of nuget.org - potentially with the direct support of the Foundation - but it would be great to make it easier for projects to find and evaluate options in terms of other packages. One thing we're often missing in the tech world across all platforms is guidance rather than just documentation. Creating guidance that is fair around multiple competing projects is difficult of course, but I'd rather try and fail than not try. (I would say that probably the single biggest improvement there would be nuget.org package search working rather better than it does now, but that's another matter.)

[schneidenbach]

@jskeet really thoughtful answer. I think you're well positioned to help solve for this problem long term. Love your vision for having the Foundation lead the way in solving for some of the more painful parts of NuGet versioning.

While I'd never particularly want to dissuade anyone from building anything new, I think it's always worth exploring what's available too. Then if you still want to start something new, at least you're doing so with your eyes open.

Couldn't have put it better myself.

[xen2]

Lot of progress has been made recently to make NuGet much more usable, covering lot of similar use cases as npm.

Maybe what is missing is a way for people to easily find and navigate projects (including similar projects).
Right now the ecosystem is quite fragmented and I wouldn't expect to find a similar project by just using NuGet, I probably would have to resort to a GitHub search or a search engine.

The reason is that nuget.org is still a very simple package database filled by package creator and without much meta-information.
However, it could be improved greatly (either nuget.org or through another meta-indexing website) by adding some more meta information (i.e. links to similar packages, etc.).
It's possible a social component might be useful too (comments, people vote for tags/similar packages, etc.).
Also having better concept of organizations/product might help when a project is fragmented in many small libraries. It could contain description about a product/org, official links, etc. Right now the nuget.org concept of owner is quite limited.

It would then become much easier to find a package that matches your needs.
And of course, if you don't like the license or features of an existing package, nothing should prevent you from creating your own.

[SeanKilleen]

I think that the simplicity of "smaller amounts of well-defined packages" has some cultural drawbacks that may be a little unique to the .NET ecosystem. For a long time, the mindset of "developer as consumer" meant that devs would take the packages provided by a central source (Microsoft in this case) and would be distrustful or dismissive of packages not "blessed" by Microsoft. As many know, Microsoft has had a long and...storied history of how it approached this issue, and there are a lot of folks with strong opinions on it.

For that reason, I think it's a positive thing to move towards a greater diversity of package offerings -- I think that it could normalize the idea of contributing to projects better moves away from the "consumer-first" or "consumer-only" mindsets.

However, I also see an issue with going the npm route in terms of tiny, fragmented packages that build massive webs of dependencies. I think as a whole the .NET ecosystem does a pretty good job of balancing these things.

In terms of new takes on existing packages vs. overcrowding the ecosystem with choices, I think we want to encourage innovation and exploration while also helping folks see that competition isn't the only way. I think there are some things that could help here:

• Highlighting packages that appear to me well-maintained & cared for, and what we think they do well, as a way of encouraging others and spreading knowledge & tips
• Making use of great resources that already exist such as Discover .NET (thanks, @daveaglick!)
• Providing some helpful guides around "so, you want to build an OSS library?" that encourages looking to see if you align with an existing project, and provides some good guidance on how to build & maintain a successful OSS library. Perhaps with contributions from successful OSS authors.
• Positioning the Foundation as a bridge for Microsoft to incorporate OSS work and avoid not-invented-here syndrome
• Ensuring that the Foundation doesn't discourage new ways of thinking or new takes on the same ideas
• Building "ecosystem explorations" where we look at choices in the ecosystem for a common task (for example, Dependency Injection) and highlight the similarities / differences in their approaches or use cases. This would allow us to highlight great work by authors, give them input into the content itself, and show a wealth of ideas in the community without favoring one specific solution

These are just some things I've thought of so far. As a Foundation with finite time & financial resources, I think we'll need to get input on items like these and choose which of these we believe can have the greatest impact and where we can affect the most positive change.

[robertmclaws]

Looking at the 1700 folders and 209MB of stuff in the "node-modules" folder for the web app in one of my projects, I would say that is not something that the .NET Community should strive to emulate. While choice is great, it is sometimes really hard to find what you're looking for in the .NET open source world of GitHub + NuGet.

I personally would suggest work with both of those teams to surface more features to help categorize existing solutions, and better usability for the NuGet experience inside IDEs to help find what you're looking for. Problems I see that cause people to spend time re-building existing solutions:

• I should be able to click on the "Package Creator" in the list in Visual Studio and pivot to the other package that user has created.
• Users should be able to rate packages on NuGet like they can rate AddIns in the VS Marketplace.
• I should be able to get a visual cue when a package in the list hasn't been updated in a long time.
• The "Manage NuGet Packages" dialog should let me expand a package and see what other libraries depend on it. That would allow people to make more informed decisions as to the validity of the library.
• There should be easier ways to contact project maintainers on GitHub and offer to take over stagnating projects.

Although I absolutely love all of @SeanKilleen's ideas in his post (for example) and would gladly get behind them if that's what the Foundation decided, I honestly feel like putting the Foundation's weight behind advocating that other teams solve their discoverability issues would go much farther (and with less effort) than giving ourselves more work to do.

[ericsink]

My first reaction is to keep the foundation neutral on whether the package ecosystem should be broader vs deeper. Both have their pros and cons. Let's not "try to control what won't".

That said, there may be interesting things if we look deeper. For example, if the trend is toward fewer packages because of a tendency for people to only trust dependencies that come from Microsoft, then that seems unhealthy, and a potential place for the foundation to show leadership.

[mairaw]

I think this is a great question and lots of interesting ideas on this thread already.
I agree that I don't think we need to dictate where someone puts their energy and effort, but with so many projects already in need of help, I'd favor getting those more discoverability and help opportunities.

What I think it needs to happen:

• Improve NuGet search. Today you can't even distinguish packages from global tools via search. Better filters as well. I think @jskeet, @SeanKilleen, and @robertmclaws already presented some nice ideas around this.
• We need a better way for the community to find opportunities to help. Navigating the sea of issues across the repos is overwhelming.
  Something similar to https://up-for-grabs.net/ but focused on .NET Foundation projects? I wasn't familiar with discover.net.
  But looking at discover.net, if I'm trying to find something to contribute, I find the site very hard to navigate for a new contributor. I think we need an issues page that shows issues where teams/projects have intentionally marked they need help (up-for-grabs/help-wanted). up-for-grabs lets you customize what is the tag you use for that. You need to be able to filter on projects, easy/good first issues, maybe distinguish Microsoft and community projects, etc.
  For the .NET documentation, we do have projects that are specific to the docs repos with community contributions ideas, such as https://github.com/dotnet/docs/projects/35. The same idea can also be created at the dotnet org level (https://github.com/orgs/dotnet/projects) and issues could be categorized into different task types and levels.

[IEvangelist]

First of all...

I know that .NET is far from JavaScript world, where NPM has more packages than space shuttle has parts.

I'm sitting here literally laughing out loud in the airport - and I'm certain that people are wondering why, if they only knew... "JavaScript" 🤣

I really like this question. I'm not certain if I would "wish .NET community would go that way" - it reminds me of the day that the NPM "left-pad" package broke the internet. I mean there are always pros and cons. It is a matter of encouragement. I think of the .NET ecosystem differently than I do the JavaScript ecosystem. I'm not sure there is really a right or wrong answer here either.

I believe in collaboration, but it can be intimidating to attempt to contribute to an existing project. I would imagine that there are ways to improve this experience. I think you're spot on with your feelings about .NET Standard and Java and the evolution of .NET. It has come a long way and open source is making it better. I think we will continue to evolve and the .NET Foundation will help cater to the needs to this evolution. I'm excited to be a part of it. As a community we should encourage developers to collaborate but at the same time we shouldn't make it hard for them to do so. Again, another area for improvement.

[duracellko]

Something similar to https://up-for-grabs.net/ but focused on .NET Foundation projects? I wasn't familiar with discover.net.
But looking at discover.net, if I'm trying to find something to contribute, I find the site very hard to navigate for a new contributor.

I agree, I didn't know discover.net before. And also find it complicated to navigate.

[mitchelsellers]

I believe that many of the others have provided great feedback on this. Many of the ideas expressed I also agree with.

The proper answer, I believe, to this question though is to make sure that we don't blend the role of the .NET Foundation with the roles & responsibilities of project leaders. The .NET Foundation, as described on the website is:

The .NET Foundation is an independent organization created to foster innovation, which we believe starts with open development and collaboration. It's also a forum for community and commercial developers to broaden and strengthen the future of the .NET ecosystem.

In furtherance of this goal, I don't believe it would be right for us to impose any arbitrary rules or suggestions as to more or less packages. However, I think the tools for collaboration and the support of individual projects might allow us a pathway to help limit the number of 'popup' libraries etc. We can do that in a number of ways.

• Providing a better mechanism for contributors to find locations/projects to contribute. This is something discussed in another question, but the key here is helping to match needs with experiences. Getting the right people to help elevate the various projects to the levels that make sense.

• Providing tooling & guidance on how to best distribute & manage your project. Discoverability is key, I believe that along with providing the tools, it is important to share success stories & processes, as well as failures to help give projects the tools to best utilize the megaphone opportunities included in NuGet/GitHub/etc.

• Providing resources to lower the administrative burden for teams to accept contributions, with a goal again of helping to prevent pop-up addtiional projects when community contributions to the main project can be done more easily.

Regardless, we want to embrace the momentum that all projects in the Foundation have and work to help to not only keep the mometum, but to grow it further. If this results in more, or less packages, I'm indifferent. I just want to see quality, supported packages.