Change Development phase to Experimental phase of a feature to better represent the state of a feature #495

jacob-delgado · 2020-11-11T17:59:39Z

As discussed in the TOC on Friday Nov 6, 2020 there is misunderstanding revolving around Experimental, Development and Pre-Alpha.

The goal is to clarify what, if any of these terms should mean. After consensus is reached and this PR is merged I will update the documentation on the istio.io site to reflect this.

Examples:

jacob-delgado · 2020-11-11T18:01:19Z

cc @dcberg @smawson @louiscryan @linsun @duderino @nrjpoddar

I wasn't able to assign this to the TOC.

FEATURE-LIFECYCLE.md

smawson · 2020-11-11T19:54:18Z

I'd like to consider just combining dev and experimental into a single phase. They're basically the same modulo whether they show up in docs. Can we just rename development to experimental and get rid of development as a separate described phase?

nrjpoddar · 2020-11-11T19:56:58Z

I don't think we currently describe development or experimental in user facing docs, so yes, we should just describe "experimental" and change all guides that say Development -> Experimental.

nrjpoddar · 2020-11-11T19:57:59Z

I don't think we currently describe development or experimental in user facing docs, so yes, we should just describe "experimental" and change all guides that say Development -> Experimental.

@jacob-delgado are you planning to update istio.io feature status after we reach consensus here for README?

nrjpoddar · 2020-11-11T19:58:55Z

Do we need 2 pages with similar content https://github.com/istio/community/blob/944c44ca912deb97c669a1ddcb9bf92af51cc308/FEATURE-LIFECYCLE.md and https://istio.io/latest/about/feature-stages/#feature-phase-definitions

nrjpoddar · 2020-11-11T20:00:27Z

If the "Recommended Use Cases" for Development/Experimental is "Don't" why are we exposing it to users?

jacob-delgado · 2020-11-11T20:08:12Z

I don't think we currently describe development or experimental in user facing docs, so yes, we should just describe "experimental" and change all guides that say Development -> Experimental.

@jacob-delgado are you planning to update istio.io feature status after we reach consensus here for README?

That is the intent. I'm just helping to facilitate this work at this point under the leadership from the TOC.

jacob-delgado · 2020-11-11T20:20:51Z

I'd like to consider just combining dev and experimental into a single phase. They're basically the same modulo whether they show up in docs. Can we just rename development to experimental and get rid of development as a separate described phase?

Done. I've incorporated Development and Experimental into "Experimental."

jacob-delgado · 2020-11-11T20:24:08Z

If the "Recommended Use Cases" for Development/Experimental is "Don't" why are we exposing it to users?

It was the same as it was Development. I've used the wording from Alpha as it fits into the concept better.

Looking at Experimental, these are largely things done to get support to end users prior to the development of a proper API. Many users are weary of using things that are not properly documented on the site (e.g. environmental variables, cli args), but an experimental document gives people enough confidence to deploy it in their environment. However, they may have know the risk in doing so as backwards compatibility is not a given and likely will change as the feature is proven there is sufficient need for a proper API.

smawson · 2020-11-12T00:20:17Z

Thanks for the updates, I like it. I'm happy to get this in and follow up with a discussion about whether we need this on istio.io and in github community or if we can just have one place point to another, but I think for now this is an improvement.

FEATURE-LIFECYCLE.md

nrjpoddar

@jacob-delgado I added few cosmetic changes but this is looking good. You might have to ping individual TOC members again to get their approval.

google-cla · 2020-11-13T23:05:10Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-11-13T23:05:13Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-11-13T23:05:25Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-11-13T23:05:38Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-11-13T23:05:39Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-11-13T23:05:51Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-11-13T23:05:52Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

jacob-delgado · 2020-11-13T23:06:17Z

@jacob-delgado I added few cosmetic changes but this is looking good. You might have to ping individual TOC members again to get their approval.

We didn't have time during the TOC to talk about this. I'll add it to the agenda for next week.

sdake

@jacob-delgado really nice work.

I added a few comments to help clarify the language and to use probability language (may) over can. There are also several other minor edit changes in this review recommended that don't affect the intent of the changes that improve clarity.

Cheers,
-steve

sdake · 2020-11-25T19:09:07Z

FEATURE-LIFECYCLE.md

    <td>Alpha</td>
    <td>Beta</td>
    <td>Stable</td>
  </tr>
  <tr>
    <td>Purpose</td>
-    <td>Active development of a feature, not for use by end users directly.</td>
+    <td> Feature is under active development and user facing APIs will most likely change. Users should deploy experimental features with extreme caution, preferably in non-production environments as newer versions can require a migration effort.</td>


most likely -> may
can -> may
newer versions -> experimental features (you are talking about the feature status and then switch context to the version...

sdake · 2020-11-25T19:12:41Z

FEATURE-LIFECYCLE.md

    <td>Used to get feedback on a design or feature or see how a tentative design performs, etc. Targeted at developers and expert users.</td>
    <td>Used to vet a solution in production without committing to it in the long term, to assess its viability, performance, usability, etc. Targeted at all users.</td>
    <td>Dependable, production hardened</td>
  </tr>
  <tr>
    <td>Stability</td>
-    <td>Unknown</td>
+    <td>May be buggy. Using the feature may expose bugs. Users must opt-in to use feature.</td>


If Experimental and Alpha have the same stability, then I'd just copy line 28.

Another approach is to take a riff off what is there, which is May be buggy. This feature may have limited testing and field experience. Not active by default.

This other approach represents a development feature as really green, (not field proven..)

sdake · 2020-11-25T19:13:10Z

FEATURE-LIFECYCLE.md

    <td>May be buggy. Using the feature may expose bugs. Not active by default.</td>
    <td>Code is well tested. The feature is safe for production use.</td>
    <td>Code is well tested and stable. Safe for widespread deployment in production.</td>
  </tr>
  <tr>
    <td>Security</td>
-    <td>Unknown</td>
+    <td>Using the feature may have obvious security vulnerabilities. Discovered vulnerabilities will not necessarily be communicated broadly.</td>


will not necessarily -> may not

sdake · 2020-11-25T19:13:26Z

FEATURE-LIFECYCLE.md

    <td>Using the feature may have obvious security vulnerabilities. Discovered vulnerabilities will not necessarily be communicated broadly.</td>
    <td>Any discovered security vulnerabilities will be publicly disclosed and patched.</td>
    <td>Any discovered security vulnerabilities will be publicly disclosed and patched.</td>
  </tr>
  <tr>
    <td>Performance</td>
-    <td>Unknown</td>
+    <td>Performance characteristics are unknown. Enabling said features may adversely affect performance.</td>


remove word said.

sdake · 2020-11-25T19:14:01Z

FEATURE-LIFECYCLE.md

    <td>Performance requirements are assessed as part of design.</td>
    <td>Performance and scalability are characterized, but may have caveats.</td>
    <td>Perf (latency/scale) is quantified and documented, with guarantees against regression.</td>
  </tr>
  <tr>
    <td>Support</td>
-    <td>None</td>
+    <td>No guarantees on backward compatibility. The feature may be dropped at any time without notice.</td>


dropped -> removed

sdake · 2020-11-25T19:17:28Z

FEATURE-LIFECYCLE.md

@@ -106,7 +106,9 @@ In addition to the basic alpha-level documentation, beta documentation includes
  </tr>
  <tr>
    <td>Upgradeability</td>
-    <td>None</td>
+    <td>The schema and semantics of an experimental feature can change in newer versions without any guarantees of preserving backwards compatibility thereby requiring configuration changes during upgrades.
+API versions can increment faster than the monthly release cadence and developers are not required to maintain multiple versions for backwards compatibility.


can -> may

sdake · 2020-11-25T19:17:54Z

FEATURE-LIFECYCLE.md

-    <td>None</td>
+    <td>The schema and semantics of an experimental feature can change in newer versions without any guarantees of preserving backwards compatibility thereby requiring configuration changes during upgrades.
+API versions can increment faster than the monthly release cadence and developers are not required to maintain multiple versions for backwards compatibility.
+Developers are recommended to increment the API version when schema or semantics change in an incompatible way</td>


recommended -> encouraged

sdake · 2020-11-25T19:18:30Z

FEATURE-LIFECYCLE.md

@@ -135,15 +137,16 @@ Test code coverage is >= 90%.
  </tr>
  <tr>
    <td>Reliability</td>
-    <td>None</td>
+    <td>The user should not expect the feature to be reliable as it is relatively untested.</td>


as it is -> may be
remove the world relatively

sdake · 2020-11-25T19:19:09Z

FEATURE-LIFECYCLE.md

    <td>Because the feature is relatively new, and may lack complete end-to-end tests, enabling the feature via a flag might expose bugs which destabilize Istio (e.g. a bug in a control loop might rapidly create excessive numbers of objects, exhausting API storage).</td>
    <td>Because the feature has e2e tests, enabling the feature should not create new bugs in unrelated features.
 Because the feature is relatively new, it may have minor bugs.</td>
    <td>High. The feature is well tested and stable and reliable for all uses.</td>
  </tr>
  <tr>
    <td>Support</td>
-    <td>None</td>
+    <td>There is no commitment from the Istio community to progress or complete the feature and


progress or complete-> improve, maintain, or complete

sdake · 2020-11-25T19:19:29Z

FEATURE-LIFECYCLE.md

@@ -153,7 +156,7 @@ Releases should simultaneously support two consecutive versions (e.g. v1beta1 an
  </tr>
  <tr>
    <td>Recommended Use Cases</td>
-    <td>Don't</td>
+    <td>In short-lived development or testing environments, geared towards soliciting early feedback from users to shape development efforts.</td>


this is nice !

google-cla · 2021-01-13T04:42:38Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

smawson · 2021-01-13T16:07:28Z

FEATURE-LIFECYCLE.md

@@ -135,15 +137,16 @@ Test code coverage is >= 90%.
  </tr>
  <tr>
    <td>Reliability</td>
-    <td>None</td>
+    <td>The user should not expect the feature to be reliable may be untested.</td>


FYI minor typo here, ... to be reliable and it may be untested.

* Use experimental as feature stage Pre-alpha/Development are deprecated in favor of Experimental (see istio/community#495). Update docs to reference this phase. * Add DNS proxying to experimental phase * Do not mix alpha and experimental * DNS Proxying is Alpha in 1.9; add to feature status page * Fix virtual machine install doc based on review * Fix linting issue

jacob-delgado requested a review from a team as a code owner November 11, 2020 17:59

google-cla bot added the cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. label Nov 11, 2020

istio-testing added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Nov 11, 2020

ericvn reviewed Nov 11, 2020

View reviewed changes

FEATURE-LIFECYCLE.md Outdated Show resolved Hide resolved

FEATURE-LIFECYCLE.md Outdated Show resolved Hide resolved

jacob-delgado changed the title ~~Add experimental phase of a feature~~ Change Development phase to Experimental phase of a feature to better represent the state of a feature Nov 11, 2020

nrjpoddar reviewed Nov 13, 2020

View reviewed changes

FEATURE-LIFECYCLE.md Outdated Show resolved Hide resolved

FEATURE-LIFECYCLE.md Outdated Show resolved Hide resolved

FEATURE-LIFECYCLE.md Outdated Show resolved Hide resolved

FEATURE-LIFECYCLE.md Outdated Show resolved Hide resolved

nrjpoddar reviewed Nov 13, 2020

View reviewed changes

FEATURE-LIFECYCLE.md Outdated Show resolved Hide resolved

nrjpoddar reviewed Nov 13, 2020

View reviewed changes

google-cla bot added cla: no Set by the Google CLA bot to indicate the author of a PR has not signed the Google CLA. and removed cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. labels Nov 13, 2020

sdake reviewed Nov 25, 2020

View reviewed changes

istio-testing added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 13, 2021

Add experimental phase of a feature

31c71aa

jacob-delgado requested review from nrjpoddar, ericvn and sdake January 13, 2021 04:43

google-cla bot added cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. and removed cla: no Set by the Google CLA bot to indicate the author of a PR has not signed the Google CLA. labels Jan 13, 2021

nrjpoddar approved these changes Jan 13, 2021

View reviewed changes

istio-testing merged commit a709b56 into istio:master Jan 13, 2021

smawson reviewed Jan 13, 2021

View reviewed changes

jacob-delgado mentioned this pull request Jan 13, 2021

Use experimental as feature stage istio/istio.io#8772

Merged

jacob-delgado mentioned this pull request Feb 4, 2021

Reconcile feature status with https://github.com/istio/community/edit/master/FEATURE-LIFECYCLE.md istio/istio.io#8903

Merged

Change Development phase to Experimental phase of a feature to better represent the state of a feature #495

Change Development phase to Experimental phase of a feature to better represent the state of a feature #495

Uh oh!

Conversation

jacob-delgado commented Nov 11, 2020

Uh oh!

jacob-delgado commented Nov 11, 2020

Uh oh!

Uh oh!

Uh oh!

smawson commented Nov 11, 2020

Uh oh!

nrjpoddar commented Nov 11, 2020

Uh oh!

nrjpoddar commented Nov 11, 2020

Uh oh!

nrjpoddar commented Nov 11, 2020

Uh oh!

nrjpoddar commented Nov 11, 2020

Uh oh!

jacob-delgado commented Nov 11, 2020

Uh oh!

jacob-delgado commented Nov 11, 2020

Uh oh!

jacob-delgado commented Nov 11, 2020

Uh oh!

smawson commented Nov 12, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nrjpoddar left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

google-cla bot commented Nov 13, 2020

Uh oh!

google-cla bot commented Nov 13, 2020

Uh oh!

google-cla bot commented Nov 13, 2020

Uh oh!

google-cla bot commented Nov 13, 2020

Uh oh!

google-cla bot commented Nov 13, 2020

Uh oh!

google-cla bot commented Nov 13, 2020

Uh oh!

google-cla bot commented Nov 13, 2020

Uh oh!

jacob-delgado commented Nov 13, 2020

Uh oh!

sdake left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

google-cla bot commented Jan 13, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nrjpoddar left a comment •

edited

Loading