Inference

• `HPEndpointBase`_

• `HPEndpoint`_

• `HPJumpStartEndpoint`_

• `HPEndpoint Configs`_

class sagemaker.hyperpod.inference.hp_endpoint_base.HPEndpointBase

Base class for HyperPod inference endpoints.

This class provides common functionality for managing inference endpoints on SageMaker HyperPod clusters orchestrated by Amazon EKS. It handles Kubernetes API interactions for creating, listing, getting, and deleting inference endpoints.

classmethod call_create_api(metadata: Metadata, kind: str, spec: _HPJumpStartEndpoint | _HPEndpoint, debug: bool = False)

Create an inference endpoint using Kubernetes API.

Parameters:

Parameter	Type	Description
metadata	Metadata	Kubernetes metadata object containing name, namespace, labels, and annotations
kind	str	Kubernetes resource kind (e.g., ‘HPJumpStartEndpoint’)
spec	Union[_HPJumpStartEndpoint, _HPEndpoint]	Endpoint specification

Raises:

Exception: If endpoint creation fails

Usage Examples

>>> from sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config import _HPJumpStartEndpoint
>>> from sagemaker.hyperpod.common.config.metadata import Metadata
>>> spec = _HPJumpStartEndpoint(...)
>>> metadata = Metadata(name="my-endpoint", namespace="default")
>>> HPEndpointBase.call_create_api(metadata, "HPJumpStartEndpoint", spec)

classmethod call_list_api(kind: str, namespace: str)

List inference endpoints using Kubernetes API.

Parameters:

Parameter	Type	Description
kind	str	Kubernetes resource kind to list
namespace	str	Kubernetes namespace to list endpoints from

Returns:

dict: List of endpoints in the specified namespace

Raises:

Exception: If listing endpoints fails

Usage Examples

>>> endpoints = HPEndpointBase.call_list_api("HPJumpStartEndpoint", "default")
>>> print(f"Found {len(endpoints['items'])} endpoints")

classmethod call_get_api(name: str, kind: str, namespace: str)

Get a specific inference endpoint using Kubernetes API.

Parameters:

Parameter	Type	Description
name	str	Name of the endpoint to retrieve
kind	str	Kubernetes resource kind
namespace	str	Kubernetes namespace containing the endpoint

Returns:

dict: Endpoint details

Raises:

Exception: If retrieving endpoint fails

Usage Examples

>>> endpoint = HPEndpointBase.call_get_api("my-endpoint", "HPJumpStartEndpoint", "default")
>>> print(endpoint['metadata']['name'])

call_delete_api(name: str, kind: str, namespace: str)

Delete an inference endpoint using Kubernetes API.

Parameters:

Parameter	Type	Description
name	str	Name of the endpoint to delete
kind	str	Kubernetes resource kind
namespace	str	Kubernetes namespace containing the endpoint

Raises:

Exception: If deleting endpoint fails

Usage Examples

>>> base = HPEndpointBase()
>>> base.call_delete_api("my-endpoint", "HPJumpStartEndpoint", "default")

classmethod get_operator_logs(since_hours: float)

Get logs from the inference operator.

Retrieves logs from the HyperPod inference operator pods for debugging and monitoring purposes.

Parameters:

Parameter	Type	Description
since_hours	float	Number of hours back to retrieve logs from

Returns:

str: Operator logs with timestamps

Raises:

Exception: If no operator pods found or log retrieval fails

Usage Examples

>>> logs = HPEndpointBase.get_operator_logs(1.0)
>>> print(logs)
>>>
>>> # Get logs from last 30 minutes
>>> logs = HPEndpointBase.get_operator_logs(0.5)

classmethod get_logs(pod: str, container: str = None, namespace=None)

Get logs from a specific pod.

Retrieves logs from a pod associated with an inference endpoint.

Parameters:

Parameter	Type	Description
pod	str	Name of the pod to get logs from
container	str, optional	Container name. If not specified, uses the first container in the pod
namespace	str, optional	Kubernetes namespace. If not specified, uses the default namespace

Returns:

str: Pod logs with timestamps

Raises:

Exception: If log retrieval fails

Usage Examples

>>> logs = HPEndpointBase.get_logs("my-pod-name")
>>> print(logs)
>>>
>>> # Get logs from specific container
>>> logs = HPEndpointBase.get_logs("my-pod", container="inference")
>>>
>>> # Get logs from specific namespace
>>> logs = HPEndpointBase.get_logs("my-pod", namespace="my-namespace")

classmethod list_pods(namespace=None)

List all pods in a namespace.

Parameters:

Parameter	Type	Description
namespace	str, optional	Kubernetes namespace to list pods from. If not specified, uses the default namespace

Returns:

List[str]: List of pod names in the namespace

Usage Examples

>>> pods = HPEndpointBase.list_pods()
>>> print(f"Found {len(pods)} pods: {pods}")
>>>
>>> # List pods in specific namespace
>>> pods = HPEndpointBase.list_pods(namespace="my-namespace")

classmethod list_namespaces()

List all available Kubernetes namespaces.

Returns:

List[str]: List of namespace names

Usage Examples

>>> namespaces = HPEndpointBase.list_namespaces()
>>> print(f"Available namespaces: {namespaces}")

class sagemaker.hyperpod.inference.hp_endpoint.HPEndpoint

Bases: _HPEndpoint, HPEndpointBase

classmethod list_pods(namespace=None, endpoint_name=None)

List all pods in a namespace.

Parameters:

Parameter	Type	Description
namespace	str, optional	Kubernetes namespace to list pods from. If not specified, uses the default namespace

Returns:

List[str]: List of pod names in the namespace

Usage Examples

>>> pods = HPEndpointBase.list_pods()
>>> print(f"Found {len(pods)} pods: {pods}")
>>>
>>> # List pods in specific namespace
>>> pods = HPEndpointBase.list_pods(namespace="my-namespace")

model_config: ClassVar[ConfigDict] = {'extra': 'ignore', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.hp_jumpstart_endpoint.HPJumpStartEndpoint

Bases: _HPJumpStartEndpoint, HPEndpointBase

validate_mig_profile(mig_profile: str, instance_type: str)

Validate if the MIG profile is supported for the given instance type.

Parameters:

• instance_type – SageMaker instance type (e.g., “ml.p4d.24xlarge”)

• mig_profile – MIG profile (e.g., “1g.10gb”)

Raises:

ValueError – If the instance type doesn’t support MIG profiles or if the MIG profile is not supported for the instance type

classmethod list_pods(namespace=None, endpoint_name=None)

List all pods in a namespace.

Parameters:

Parameter	Type	Description
namespace	str, optional	Kubernetes namespace to list pods from. If not specified, uses the default namespace

Returns:

List[str]: List of pod names in the namespace

Usage Examples

>>> pods = HPEndpointBase.list_pods()
>>> print(f"Found {len(pods)} pods: {pods}")
>>>
>>> # List pods in specific namespace
>>> pods = HPEndpointBase.list_pods(namespace="my-namespace")

model_config: ClassVar[ConfigDict] = {'extra': 'ignore', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Dimensions

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.CloudWatchTrigger

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

CloudWatch metric trigger to use for autoscaling

class sagemaker.hyperpod.inference.config.hp_endpoint_config.CloudWatchTriggerList

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.PrometheusTrigger

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Prometheus metric trigger to use for autoscaling

class sagemaker.hyperpod.inference.config.hp_endpoint_config.PrometheusTriggerList

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.AutoScalingSpec

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Kubernetes

Bases: BaseModel

User-provided customizations for the inference pod.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.NodeSelectorRequirement

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.NodeSelectorTerm

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.PreferredSchedulingTerm

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.NodeSelector

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.NodeAffinity

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.CustomCertificateConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Probe

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Probes

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.RequestLimits

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.IntelligentRoutingSpec

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for intelligent routing This feature is currently not supported for existing deployments. Adding this configuration to an existing deployment will be rejected.

class sagemaker.hyperpod.inference.config.hp_endpoint_config.L2CacheSpec

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for providing L2 Cache offloading

class sagemaker.hyperpod.inference.config.hp_endpoint_config.KvCacheSpec

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for KV Cache specification By default L1CacheOffloading will be enabled

class sagemaker.hyperpod.inference.config.hp_endpoint_config.LoadBalancer

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for Application Load Balancer

class sagemaker.hyperpod.inference.config.hp_endpoint_config.ModelMetrics

Bases: BaseModel

Configuration for model container metrics scraping

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Metrics

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for metrics collection and exposure

class sagemaker.hyperpod.inference.config.hp_endpoint_config.FsxStorage

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.S3Storage

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.TokenSecretRef

Bases: BaseModel

Reference to a Kubernetes Secret containing the HuggingFace API token.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.HuggingFaceModel

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

HuggingFace model configuration. Required when modelSourceType is huggingface.

class sagemaker.hyperpod.inference.config.hp_endpoint_config.ModelSourceConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Tags

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.TlsConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configurations for TLS

class sagemaker.hyperpod.inference.config.hp_endpoint_config.ConfigMapKeyRef

Bases: BaseModel

Selects a key of a ConfigMap.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.FieldRef

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

supports metadata.name, metadata.namespace, metadata.labels['<KEY>'], metadata.annotations['<KEY>'], spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.

Type:

Selects a field of the pod

class sagemaker.hyperpod.inference.config.hp_endpoint_config.ResourceFieldRef

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.

Type:

Selects a resource of the container

class sagemaker.hyperpod.inference.config.hp_endpoint_config.SecretKeyRef

Bases: BaseModel

Selects a key of a secret in the pod’s namespace

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.ValueFrom

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Source for the environment variable’s value. Cannot be used if value is not empty.

class sagemaker.hyperpod.inference.config.hp_endpoint_config.EnvironmentVariables

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

EnvVar represents an environment variable present in a Container.

class sagemaker.hyperpod.inference.config.hp_endpoint_config.ModelInvocationPort

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Defines the port at which the model server will listen to the invocation requests.

class sagemaker.hyperpod.inference.config.hp_endpoint_config.ModelVolumeMount

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Defines the volume where model will be loaded

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Claims

Bases: BaseModel

ResourceClaim references one entry in PodSpec.ResourceClaims.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Resources

Bases: BaseModel

Defines the Resources in terms of CPU, GPU, Memory needed for the model to be deployed

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Worker

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Details of the worker

class sagemaker.hyperpod.inference.config.hp_endpoint_config.CaptureContentTypeHeader

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for how to treat different content type headers during capture

class sagemaker.hyperpod.inference.config.hp_endpoint_config.CaptureOptions

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

CaptureOption defines what data to capture (input, output, or both).

class sagemaker.hyperpod.inference.config.hp_endpoint_config.BufferConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for buffering and flushing captured data

class sagemaker.hyperpod.inference.config.hp_endpoint_config.PayloadConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for payload size limits

class sagemaker.hyperpod.inference.config.hp_endpoint_config.DataCaptureModelPod

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for Model Pod level data capture (Tier 3)

class sagemaker.hyperpod.inference.config.hp_endpoint_config.DataCaptureSagemakerEndpoint

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for SageMaker Endpoint level data capture (Tier 1)

class sagemaker.hyperpod.inference.config.hp_endpoint_config.DataCaptureLoadBalancer

Bases: BaseModel

Configuration for LoadBalancer level data capture (Tier 2)

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.DataCapture

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for data capture across multiple tiers (SageMaker, LoadBalancer, Model Pod)

class sagemaker.hyperpod.inference.config.hp_endpoint_config.DnsConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

DNS automation configuration for Route53. Requires tlsConfig.customCertificateConfig to be set.

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Conditions

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

DeploymentCondition describes the state of a deployment at a certain point.

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Status

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Status of the Deployment Object

class sagemaker.hyperpod.inference.config.hp_endpoint_config.DeploymentStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Details of the native kubernetes deployment that hosts the model

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Sagemaker

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Status of the SageMaker endpoint

class sagemaker.hyperpod.inference.config.hp_endpoint_config.Endpoints

Bases: BaseModel

EndpointStatus contains the status of SageMaker endpoints

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.ModelMetricsStatus

Bases: BaseModel

Status of model container metrics collection

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_endpoint_config.MetricsStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Status of metrics collection

class sagemaker.hyperpod.inference.config.hp_endpoint_config.TlsCertificate

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

CertificateStatus represents the status of TLS certificates

class sagemaker.hyperpod.inference.config.hp_endpoint_config.DnsStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Status of the operator-managed Route53 DNS record

class sagemaker.hyperpod.inference.config.hp_endpoint_config.InferenceEndpointConfigStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

ModelDeploymentStatus defines the observed state of ModelDeployment

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Dimensions

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.CloudWatchTrigger

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

CloudWatch metric trigger to use for autoscaling

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.PrometheusTrigger

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Prometheus metric trigger to use for autoscaling

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.CloudWatchTriggerList

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.PrometheusTriggerList

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.AutoScalingSpec

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.EnvironmentVariables

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.ModelMetrics

Bases: BaseModel

Configuration for model container metrics scraping

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Metrics

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for metrics collection and exposure

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.AdditionalConfigs

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Model

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.SageMakerEndpoint

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Validations

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Server

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.IntelligentRoutingSpec

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for intelligent routing

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.L2CacheSpec

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.KvCacheSpec

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.LoadBalancer

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.CustomCertificateConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.TlsConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.CaptureContentTypeHeader

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for how to treat different content type headers during capture

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.CaptureOptions

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

CaptureOption defines what data to capture (input, output, or both).

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.BufferConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for buffering and flushing captured data

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.PayloadConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for payload size limits

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DataCaptureModelPod

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for Model Pod level data capture (Tier 3)

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DataCaptureSagemakerEndpoint

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for SageMaker Endpoint level data capture (Tier 1)

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DataCaptureLoadBalancer

Bases: BaseModel

Configuration for LoadBalancer level data capture (Tier 2)

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DataCapture

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for data capture across multiple tiers (SageMaker, LoadBalancer, Model Pod)

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DnsConfig

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

DNS automation configuration for Route53.

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Conditions

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

DeploymentCondition describes the state of a deployment at a certain point.

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Status

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Status of the Deployment Object

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DeploymentStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Details of the native kubernetes deployment that hosts the model

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Sagemaker

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Status of the SageMaker endpoint

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.Endpoints

Bases: BaseModel

EndpointStatus contains the status of SageMaker endpoints

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.ModelMetricsStatus

Bases: BaseModel

Status of model container metrics collection

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.MetricsStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Status of metrics collection

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.TlsCertificate

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

CertificateStatus represents the status of TLS certificates

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DataCaptureModelPodStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Health status of the model pod data capture tier

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DataCaptureStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Health status of the data capture pipeline

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.DnsStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

Status of the operator-managed Route53 DNS record

class sagemaker.hyperpod.inference.config.hp_jumpstart_endpoint_config.JumpStartModelStatus

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}

ModelDeploymentStatus defines the observed state of ModelDeployment