Add developer testing design proposal #10386
Conversation
|
@samcragg I'm sorry that this slipped below my radar :( What you've described is functionality similar to "fallback folders", with the exception that fallback folders have lower precedence than the global packages folder. I can think of at least two differences. First, only PackageReference supports fallback folders, not packages.config. Second, packages are not deleted from the global packages folder when they exist in the fallback folder. Instead, they're just used from the fallback folder, in the same way that packages in the global packages folder are used. Given the scenario for this proposed feature is for new package/version development and testing, it does not seem to be a valid scenario that the package should already exist in the global packages folder. Therefore, I propose that this design's outcome could be achieved with existing features:
Currently, there is no Would this workflow make this proposal redundant? @zkat as you're the current pack area owner, I'm adding assigning this to you for community PR shepherding. Please help us avoid forgetting about this again. |
zivkan
added
the
Community
|
Thanks for the feedback, if no changes are required then all the better! Just zooming out from what we (as in me and the other developers in the linked issues in the proposal) are trying to achieve is to be able to test a package before publishing it. This needs to take into account that a version of the package already exists on nuget.org and that the exact version number cannot be specified in the project used to test it (we want to test the package we just built and not have to keep updating a version number in a project every time we build). If I understand your work around then I don't think it would meet that criteria, as you need to specify the version of the package that was just built. It also requires quite a bit of effort to manually keep the fallback folder with the correct files in it. It also means immediately after publishing a package on nuget you would then have to make another commit to bump the version number again to avoid the locally built package being ignored because it has the same version as the nuget.org one. I feel this will fit in nicely with the Improve inner-loop performance for .NET developers theme for .NET 6. At the moment, there are a handful of NuGet issues all around the same theme of using the local version of the package over NuGet and none of them have a satisfactory solution yet. If there really is a lack of desire for any changes in this area to help package developers, could a sample be created and documented of the best practices for testing a package locally? |
There was a problem hiding this comment.
Thanks for starting this discussion!
Just zooming out from what we (as in me and the other developers in the linked issues in the proposal) are trying to achieve is to be able to test a package before publishing it. This needs to take into account that a version of the package already exists on nuget.org and that the exact version number cannot be specified in the project used to test it (we want to test the package we just built and not have to keep updating a version number in a project every time we build).
I wonder if adding PackageReference metadata that points to the local nupkg file is a better alternative?
NuGet could expand that in the obj directory instead of the global packages folder. Changing things in the global packages folder doesn't make it project local anymore.
| in the context of trying to test a newly created package: | ||
|
|
||
| * NuGet caches packages by ID and version | ||
| * The order of multiple sources are searched through is non-deterministic |
There was a problem hiding this comment.
https://docs.microsoft.com/en-us/nuget/concepts/package-installation-process covers this.
File feeds are preferred over http feeds.
There was a problem hiding this comment.
True if the version is specified, however, when no version is specified then both are still checked. See #10367
| searching any other sources. | ||
| * Otherwise, normal resolution happens. | ||
|
|
||
| When the override package is used, it is restored to the cache as normal (taking |
There was a problem hiding this comment.
Note that no-cache has nothing to do with the global packages folder, which is an installation directory.
It refers to the http-cache.
There was a problem hiding this comment.
Here I was trying to refere to the packages folder (either global or where specifed by the --packages option). Is there a better term for this (packages folder perhaps, though that could be the taken as the source packages folder)?
There was a problem hiding this comment.
When you specify --packages you are overriding the global packages folder value, so I think using global packages folder and packages folder in this context is good.
| etc). This prevents changes to the rest of the restore process and doesn't | ||
| affect being able to publish a different package of the same version to the | ||
| override folder, as the version will be automatically cleaned during each | ||
| restore (i.e. although it's installed in the cache, it will be removed from it |
There was a problem hiding this comment.
Given that restore is an implicit gesture in so many workflows, this could be un-performant.
Restore has a quick and cheap concept of no-op where the up to dateness of the project is affected by project versions + extracted packages only (warm VS optimistically skips the extracted packages check).
Alternatively it could be something that it neatly implemented based on pacakge timestamps? I'd need to mull over this.
There was a problem hiding this comment.
Good point, would it help if I limited the scope that this solution is only meant to allow testing of a local package (i.e. should be used as part of a build, package, test-pacakge cycle)?
| (i.e. there cannot be multiple override folders to avoid ordering issues). Note | ||
| that file shares _should_ be supported, however. | ||
|
|
||
| Also, only a single version of a package (based on its ID) may exist in the |
There was a problem hiding this comment.
Is the idea to overwrite many packages at once?
There was a problem hiding this comment.
How do we handle transitive dependencies?
Would those be expected to be in that local folder?
I think package versions have to matter for transitive dependencies, because this behavior breaks when you have 3rd party dependencies to the package being developed.
There was a problem hiding this comment.
I was hoping this would be handled for free by limiting it to a single version of that package in the hypothetical override folder. When looking for dependencies, the same logic would apply, look in the override folder first and then fallback to others. I don't think versioning would matter in this case - if you've specified a version and it doesn't exist in the override folder then the normal logic applies. If you specify a version that does exist in the override folder, then that is used without checking any other sources. If you're not using an exact version but the version that exists in the override folder exists fits the bill, it will be used.
Bear in mind the override folder is really only meant to be used for testing a package that you've just built locally. I don't think other uses should be supported as it will add to complexity. But obviously, my opinion is biased 🙂
| affect being able to publish a different package of the same version to the | ||
| override folder, as the version will be automatically cleaned during each | ||
| restore (i.e. although it's installed in the cache, it will be removed from it | ||
| again the next time the restore process starts). |
There was a problem hiding this comment.
I think a neater approach instead of deleting packages would be to have an additional folder where these packages are consumed from.
Deleting packages from the global packages folder automatically can be tricky because multiple projects share the global packages folder.
There was a problem hiding this comment.
Yep, that sounds good. I personally use a different folder to restore to (via the --packages option), so we could further limit the override to only work under that scenario if it helps?
|
|
||
| | Property | Description | | ||
| |-----------------------|--------------| | ||
| | RestoreOverrideFolder | Specifies a folder that contains `nupkg` files that take precedence over any other source and are not cached. | |
There was a problem hiding this comment.
The one concern I have with a project, instead of a package metadata driven solution is this doesn't work transitively.
Not sure it should be a complete blocker, but it is something that's a concern.
There was a problem hiding this comment.
To be honest, I'll let you lot specify the exact mechanism 🙂 This way seemed the least impacting, but if you want another option when running the restore command or to have it in a separate file then I certainly wouldn't be offended.
Claiming there's a "lack of desire for any changes" is a completely unfair characterization. Feels like a strawman argument. I'm trying to have a conversation to see how we can improve the spec/feature/workflow/experience, rather than accepting the first proposal as-is. The spec doesn't say what the expected "end-to-end" workflow would be, but if you want to be able to edit a line of code in the product, and then run This proposal only works in the "edit code, then run test" scenario if the build chain is changed to restore projects after project references are compiled, which would be a massive design change in NuGet and would have significant performance impact given PackageReference's transitive nature, not just on packages, but also project references. Being able to restore all projects in the solution in parallel is a significant performance gain. If we're only trying to improve "edit code, pack nupkg, then run test" workflow, then we can investigate whether fallback folders, something equivalent to Regarding not needing to update version numbers in the test project all the time, I believe there are existing features, at least for PackageReference. How ever you're generating the version number for the package version, whether that's |
|
cc @JonDouglas @anangaur for visibility :) |
|
Thanks for the response @zivkan . First of all, appologies if my comment came off as argumentative, that wasn't my intention. To clarify, if no changes are required to NuGet than all the better as far as I'm concerned 🙂 That sentance was my attempt to offer an olive branch of what I would consider an acceptable alternative solution; since there have been multiple issues raised around a similar theme and they all have workarounds proposed (including this PR has a proposed workaround), I would happily accept just a sample project that I could look at that showcases existing functionality. Hope that helps to clairify a little. For your question around the scenario I'm considering, it's specifically there doesn't appear to be a good way of doing the "edit code, pack nupkg, then run test" cycle. This is what I really want to achieve - given a package that I've just produced, can I be certain that when an end user restores it that the relavant MSBuild tasks that I've packaged will do the correct thing on the destination project? Would it help if I created a minimal solution to demonstrate this? I appreciate it is quite a niche requirement, but it does appear I'm not the only one wanting to do this. |
|
Team Review: How much does this new package provide in this area? |
|
I think that feature is more complementary instead of replacing the ask here. |
|
This PR has been automatically marked as stale because it has no activity for 30 days. It will be closed if no further activity occurs within another 330 days of this comment. If it is closed, you may reopen it anytime when you're ready again, as long as you don't delete the branch. |
No description provided.