A/B testing

Measure the impact of the community on your app by giving access to only part of your audience.

Before you begin

• Complete the Quickstart: the SDK is initialized.
• Choose who runs the test: Octopus (an Octopus-run A/B test), or your own A/B testing tool.

How it works

An A/B test splits your audience into two cohorts:

• Test group: a part of your users (for example 30%) with full access to the community.
• Control group: the other users (for example 70%), without access.

Octopus Analytics then compares the two cohorts on app session volume and retention.

There are two ways to run the test:

	Octopus-run A/B test	Your own A/B test
Who assigns the cohort	Octopus	Your app
What the control group sees	A screen saying the community is not available yet for them	Whatever your app decides: usually no community entry point
What your app does	Reads the user's access (step 1), and optionally overrides it (step 2)	Reports the user's cohort to Octopus (step 3)
Effect on the SDK	Access is enforced by the SDK	Analytics only: the SDK behaves the same for both cohorts

---

1. Read whether the user has access

Android availableiOS availableFlutter availableReact Native availableUnity ≥ 1.12.6

With an Octopus-run test, read whether the current user is in the test group, for example to show or hide your community entry point. The value is updated as soon as the user's cohort changes.

Android

• OctopusSDK.hasAccessToCommunity: Flow<Boolean> — emits the current access, then every change.

OctopusSDK.hasAccessToCommunity.collect { hasAccessToCommunity ->
    // Show or hide your community entry point
}

iOS

• octopus.hasAccessToCommunity: Bool — a @Published property: read it once, or observe $hasAccessToCommunity.

// One-shot value
let hasAccess = octopus.hasAccessToCommunity

// Published value
octopus.$hasAccessToCommunity
    .sink { hasAccessToCommunity in
        // Show or hide your community entry point
    }
    .store(in: &cancellables)

Flutter

• OctopusSDK.hasAccessToCommunity: Stream<bool> — replays the current access, then every change.

final subscription = OctopusSDK.hasAccessToCommunity.listen((hasAccessToCommunity) {
  // Show or hide your community entry point
});

React Native

• addHasAccessToCommunityListener(callback) — calls callback with the access whenever it changes. Returns a subscription to remove().

import { addHasAccessToCommunityListener } from '@octopus-community/react-native';

const subscription = addHasAccessToCommunityListener((hasAccessToCommunity) => {
  // Show or hide your community entry point
});

// Later, to unsubscribe
subscription.remove();

Unity

• OctopusSDK.HasAccessToCommunity: bool — the cached value, for a one-shot read.
• OctopusSDK.OnHasAccessToCommunityChanged: Action<bool> — raised when the access changes.

// One-shot value
bool hasAccess = OctopusSDK.HasAccessToCommunity;

// Changes
OctopusSDK.OnHasAccessToCommunityChanged += hasAccessToCommunity =>
{
    // Show or hide your community entry point
};

---

2. Override the user's access

Android availableiOS availableFlutter availableReact Native availableUnity ≥ 1.12.6

Override the cohort Octopus assigned to the current user: to check what each cohort sees during your tests, or to grant or remove access for given users. The override takes precedence over the Octopus-run test and over the cohort you report in step 3. Once it is applied, the access of step 1 emits the new value.

Android

• hasAccess: Boolean — true to grant access, false to remove it.
• Returns OctopusResult<Unit, OverrideCommunityAccessError>; the function is suspend.

scope.launch {
    when (val result = OctopusSDK.overrideCommunityAccess(hasAccess = canAccessCommunity)) {
        is OctopusResult.Success -> {
            // Override applied: hasAccessToCommunity emits the new value
        }
        is OctopusResult.Failure -> {
            // No network, user not connected, override refused…
        }
    }
}

iOS

• access: Bool — true to grant access, false to remove it.
• The function is async throws.

Task {
    do {
        try await octopus.overrideCommunityAccess(canAccessCommunity)
        // Override applied: hasAccessToCommunity publishes the new value
    } catch {
        // No network, user not connected, override refused…
    }
}

Flutter

• hasAccess: bool — true to grant access, false to remove it.
• Returns Future<OctopusResult<void, OverrideCommunityAccessError>>.

final result = await octopus.overrideCommunityAccess(canAccessCommunity);
switch (result) {
  case OctopusSuccess():
    // Override applied: hasAccessToCommunity emits the new value
    break;
  case OctopusInvalidArguments<OctopusServerError>(:final errors):
    // Naming `<OctopusServerError>` keeps the switch exhaustive.
    // The only OverrideCommunityAccessError is OverrideCommunityAccessUnknownError.
    for (final error in errors.whereType<OverrideCommunityAccessError>()) {
      debugPrint(error.errorMessage);
    }
  case OctopusConnectionFailure():
    // No network, user not connected…
}

React Native

• hasAccess: boolean — true to grant access, false to remove it.
• Returns Promise<void>; a failure rejects with an error you can test with isOverrideCommunityAccessError (code OVERRIDE_ERROR).

import { overrideCommunityAccess, isOverrideCommunityAccessError } from '@octopus-community/react-native';

try {
  await overrideCommunityAccess(canAccessCommunity);
  // Override applied: addHasAccessToCommunityListener emits the new value
} catch (error) {
  if (isOverrideCommunityAccessError(error)) {
    console.warn(`${error.code}: ${error.message}`);
  }
}

Unity

• hasAccess: bool — true to grant access, false to remove it.
• onCompleted: Action and onError: Action<string> — optional, called on the Unity main thread; onError receives the error message.

OctopusSDK.OverrideCommunityAccess(
    canAccessCommunity,
    onCompleted: () => { /* Override applied: OnHasAccessToCommunityChanged fires */ },
    onError: message => Debug.LogError($"Community access override failed: {message}"));

---

3. Report your own A/B test cohort

When your app runs its own A/B test, tell the SDK whether the current user is in the community-enabled group or in the control group, so that Octopus Analytics can compare them. This only feeds analytics: it does not change what the SDK shows.

The value is reset at each SDK launch: report it every time, as soon as possible after the SDK is initialized.

Android

• hasAccess: Boolean? — true for the community-enabled group, false for the control group.

OctopusSDK.trackAccessToCommunity(hasAccess = canAccessCommunity)

iOS

• hasAccessToCommunity: Bool — true for the community-enabled group, false for the control group.

octopus.track(hasAccessToCommunity: canAccessCommunity)

Flutter

• hasAccess: bool — true for the community-enabled group, false for the control group.

await octopus.trackCommunityAccess(canAccessCommunity);

React Native

• hasAccess: boolean — true for the community-enabled group, false for the control group.

import { trackCommunityAccess } from '@octopus-community/react-native';

await trackCommunityAccess(canAccessCommunity);

Unity

• hasAccess: bool — true for the community-enabled group, false for the control group.

OctopusSDK.TrackAccessToCommunity(canAccessCommunity);

---

See it in the samples

Android

The Octopus Sample app.

iOS

The Force Octopus A/B Tests Cohort and Track A/B Tests scenarios.

Flutter

The force Octopus A/B tests and track A/B tests scenarios of the example app.

React Native

The scenarios screen of the example app.

Unity

The ForceOctopusABTestsScenario and TrackABTestsScenario scenarios of the Unity example.

Behavior and limits

• An override (step 2) takes precedence over both the Octopus-run test and the cohort you report (step 3).
• The reported cohort is reset at each SDK launch: report it again after every initialization.
• The reported cohort is used for analytics only. To actually hide the community from the control group of your own test, hide your entry point.

Next steps

• Analytics — send custom events and listen to the events of the SDK.
• Events reference — every event the SDK emits.