DAVIS — Depth-Only Active Vision for Humanoid Soccer

Abstract

The paper's question, interface, learning structure, and evidence in one place.

Humanoid soccer contact skills require more than producing high-impact foot–ball contacts: the robot must close the loop over perception, approach, alignment, impact, and recovery while its own motion induces substantial viewpoint changes, frequent loss of the ball from view, and uncertain contact outcomes. In this work, we ask a compact yet stricter question: can a humanoid learn soccer contact skills using only a head-mounted depth image, proprioceptive history, and an optional low-dimensional task command, and directly output 25-DoF joint PD targets without extra runtime perception or planning modules? To this end, we propose DAVIS, a depth-only end-to-end framework for humanoid soccer skills that learns visibility-aware auxiliary geometry during training, and combines GT-to-prediction annealing, task curricula, and AMP-style motion priors to smoothly bridge privileged supervision and real deployment. Built on this framework, we instantiate representative soccer contact skills, including goal-directed shooting and directional dribbling, through task-specific definitions of objects, commands, rewards, and curricula, and validate them through simulation, Noetix E1 real-robot experiments, and ablations.

Overview

Why soccer contact needs active vision, and what DAVIS removes from the runtime stack.

A humanoid soccer policy has to perceive a moving ball, approach it, align the body and foot, make contact, and recover into the next behavior. The camera is mounted on the actuated head, so locomotion and contact continuously change the next observation. A fixed gaze can therefore lose the ball or goal exactly when precise contact is needed. DAVIS treats head motion as part of the control problem rather than as a detached look-at-ball module.

The deployment question is deliberately strict: can the robot infer local task geometry from its own depth image and proprioception, then output whole-body targets without an external detector, localization system, state estimator, or soccer planner? DAVIS puts structure in the training signal while keeping the exported runtime graph small.

Depth-only control

One deployable graph

One aligned depth frame, a five-step proprioceptive history, and an optional low-dimensional command are mapped to a 25-dimensional joint residual. No scene state is injected at runtime.

Active vision

The head is part of the loop

The E1 has 23 body DoFs and 2 head DoFs. Those head actions change the next depth frame, so keeping task objects visible is learned jointly with approach and contact.

Structure inside training

Geometry with confidence

Privileged simulation labels supervise geometry and visibility heads, while curricula, GT-to-prediction annealing, and AMP-style priors make the final depth-only policy usable.

DAVIS teaser showing shooting, dribbling, and active vision — **DAVIS in action.** Goal-directed shooting, directional dribbling, and active camera control on the Noetix E1.

Video

Method

Inputs, outputs, auxiliary geometry, and the training-to-deployment transition.

DAVIS models soccer as a partially observed control problem. During training, the critic and auxiliary losses may use simulator state; the actor-side task geometry must eventually come from the same depth/proprioception interface available on the robot.

Runtime interface. The actor receives a single aligned 168 × 80 depth frame, a five-step proprioceptive history, and an optional task command. It outputs 25 joint-position residuals, converted to q^des = q⁰ + s ⊙ a by the low-level PD controller. The 25 DoFs comprise 23 body joints and 2 active head joints.

Observation

What the actor sees

Depth is clipped to 0.3–5.0 m, normalized as 2D/5.0 − 1, and zero-filled at invalid pixels. Temporal context comes from proprioceptive history rather than stacked depth frames. Shooting uses 81-D proprioception; dribbling uses 84-D.

Representation

Depth plus body history

LightDepthEncoder produces a 32-D depth latent. A HIM feature encodes proprioceptive history and is concatenated with proprioception, command, and confidence-gated object features before the actor MLP.

Asymmetric learning

Privileged critic, deployable actor

Base velocity, contacts, terrain scan, object state, visibility labels, and curriculum variables help value learning and supervision in simulation; they are not runtime actor inputs.

DAVIS depth-only structured reinforcement learning framework — **Structured depth and asymmetric training.** PPO, a privileged critic, HIM history encoding, geometry supervision, visibility gating, curricula, and AMP-style priors shape one deployable policy graph.

Visibility-aware auxiliary geometry

For each task object, an auxiliary head predicts a 3-D center and a visibility logit from the current depth latent. Coordinates are in the right-handed head-yaw-aligned frame: x forward, y left, z up. Geometry loss is applied only when the object is visible; visibility is trained with binary cross-entropy. The predicted center is multiplied by a detached confidence gate clipped to [0.05, 1], so uncertain geometry contributes less to the actor without allowing policy gradients to manipulate the gate.

GT → prediction annealing. Early updates blend privileged simulator geometry with predicted center and visibility. A monotone square-root schedule moves the actor toward prediction-only features and can pause when the auxiliary-loss EMA is too high. At deployment the blend is 100% predicted from depth.

Visibility-aware auxiliary geometry mechanism — **Scalable auxiliary geometry.** Each task object receives geometry and visibility heads; confidence-gated outputs extend from one object to n objects without adding a separate runtime perception module.

Task definitions

One observation-to-control interface; different objects, commands, rewards, and curricula.

A skill is specified through task objects, an optional command, task geometry, rewards, and a curriculum. Shooting and dribbling are trained as separate checkpoints, not a single multi-skill policy; they retain the same depth-to-control interface.

Object set: ball + goal

Goal-directed shooting

The ball-to-goal direction defines a behind-ball approach target. The policy receives no operator command and learns run-up, head tracking, body alignment, impact, goal-directed exit motion, and post-contact stabilization. Simulation covers penalty kicks (robot 2–3.5 m behind the ball, −100° to +100°) and free kicks (robot 5 m in front of goal; ball x ∈ [0,4] m, y ∈ [−3,3] m).

Object set: ball

Directional dribbling

A joystick supplies only a 12-way direction command c = kπ/6. The policy keeps the ball in a controllable band (robot–ball target 0.42 m, ball speed 0.20–1.50 m/s, command-aligned velocity) while making progress. Repeated-S slalom, target reaching, and obstacle traversal test sustained regulation rather than one high-speed strike.

Qualitative examples of DAVIS soccer skills in simulation and on the real robot — **Representative skills.** Curved and penalty kicks, cone-avoidance and field dribbling, stopping a rolling ball, and real-robot foot–ball contact. Appendix F details obstacle-aware dribbling and ball-loss recovery; the latter is demonstrated in the video.

Results

Simulation coverage, real-robot transfer, and controlled evidence for the training structures.

Separate shooting and dribbling policies share the same depth-to-control interface. The evaluation asks whether depth-only contact works, which training structures are necessary, and whether the learned policies transfer to the Noetix E1.

Free-kick and penalty-kick success regions with shooting footsteps in simulation — **Shooting in simulation.** Free-kick and penalty-kick success regions, each evaluated with more than 50 trials, with representative footstep trajectories.

Goal-directed shooting

Task	Easy	Medium	Hard	Total
Penalty kick SR	0.95	0.96	0.56	0.85
Free-kick SR	1.00	0.88	0.71	0.85
Real penalty-kick SR	0.68 (26/38)	0.57 (28/49)	0.55 (33/60)	—

Repeated-S slalom statistics and foot-contact record in simulation — **Repeated-S slalom.** Statistics from 100 S-slalom trajectories and the foot-contact record of a representative successful trajectory.

Directional dribbling

The repeated-S benchmark crosses three pole counts with three turn angles. Each cell is evaluated with 100 trials (900 in total); the overall mean SR is 0.65.

Metric	N = 3			N = 5			N = 7
Metric	60°	90°	120°	60°	90°	120°	60°	90°	120°
SR ↑	0.71	0.77	0.65	0.66	0.72	0.55	0.68	0.64	0.43
E_vel ↓	0.69	0.74	0.91	0.74	0.73	0.80	0.71	0.81	0.97
R_bib ↑	0.73	0.74	0.64	0.76	0.75	0.70	0.76	0.72	0.60

Real-robot dribbling records 0.79 (22/28) for target reaching, 0.68 (19/28) for obstacle traversal, and 0.61 (17/28) for slalom. Success requires the robot to remain upright and both robot and ball to finish within 1 m of the target.

Simulation ablations

All variants use the same training parameters and evaluation settings. Shooting columns report SR, angular error in degrees, and valid-contact ratio in percent; slalom columns report SR, velocity error, and ball-in-band ratio.

Method	Penalty kick			Free kick			Repeated-S slalom
Method	SR ↑	E_ang ↓	r_contact ↑	SR ↑	E_ang ↓	r_contact ↑	SR ↑	E_vel ↓	R_bib ↑
DAVIS	0.84	23.29	99.75	0.80	30.18	99.27	0.70	0.79	0.74
w/o curriculum	0.62	27.40	32.53	0.55	34.42	87.67	0.51	0.77	0.69
Auxiliary head w/o anneal	0.32	54.79	84.87	0.22	76.20	87.92	0.31	0.83	0.66
Auxiliary head w/o geometry	0.20	53.53	87.33	0.15	69.47	78.13	0.34	0.91	0.54
Auxiliary head w/o supervision	0.13	81.87	93.33	0.14	93.66	91.67	0.30	0.95	0.48
Auxiliary head w/o visibility gate	0.34	51.97	91.33	0.29	68.44	92.08	0.27	1.06	0.64
w/o auxiliary head	0	—	0	0	—	0	0	1.45	0.05
w/o active vision	0	—	0	0.36	97.81	1.71	0	1.25	0.03
Heuristic head controller	0.88	26.66	93.67	0.76	37.96	88.13	0	0.99	0.13

The 0.70 slalom SR in the ablation table comes from its controlled ablation set; the 0.65 mean above is from the separate 900-trial benchmark. Removing the auxiliary head drives all three tested success rates to zero; fixing the head drives penalty and slalom success to zero and reduces free-kick success to 0.36. Removing curriculum consistently reduces success, while a heuristic look-at-ball head remains competitive on static penalty kicks but fails on dynamic dribbling.

External protocol

Under the reproduced Dribble Master protocol, DAVIS records 0.88 target-reaching success at 3 m and 0.96 single-obstacle success at 5 m under the 1 m tolerance, using 50 simulation rounds per condition.

Deployment

How the training interface is aligned with the real Noetix E1 control stack.

TrainingIsaac Sim 5.0.0Isaac Lab · 8× RTX 4090D · 20,000 PPO iterations

Input168×80 depthFive-step proprioception · optional task command

Output25-DoF targetsJoint-position residuals passed to PD control

Runtime50 Hz actor200 Hz PD · 15 Hz depth stream

The simulator renders calibrated ZED2i distance-to-image-plane depth at 168×94, then crops it to 168×80. Matched cropping, sensor-noise perturbations, random masks, camera-pose and dynamics randomization, and identical normalization reduce the sim-to-real gap before ONNX export.

Depth alignment for real-world deployment — **Deployment bridge.** Both training and robot execution use the same single-frame depth preprocessing. No external localization, object detector, state estimator, or runtime soccer planner is used.

What the robot trials show

All real-world experiments run on grass with the onboard ZED2i mounted to the 2-DoF head. Shooting evaluates 12 predefined positions, with 26/38, 28/49, and 33/60 successful one-kick goal crossings in Easy, Medium, and Hard settings. Dribbling evaluates target reaching, obstacle traversal, and slalom with one joystick operator; safety uses a tether or staff member plus emergency-stop and projected-gravity fall triggers. If the ball is lost without the learned recovery behavior, the contact policy exits to the default walk–run policy.

Technical supplement

Experimental protocols, policy settings, reward definitions, and deployment details corresponding to the paper appendix.

Robot system and deployment interface

The training and deployment graph keeps the same 25-dimensional action interface.

TrainingIsaac Sim 5.0.0Isaac Lab · 8× RTX 4090D · 20,000 PPO iterations

Input168×80 depthFive-step proprioception · optional task command

Output25-DoF targetsJoint-position residuals passed to PD control

Runtime50 Hz actor200 Hz PD · 15 Hz depth stream

Depth is clipped to 0.3–5.0 m, normalized as 2D/5.0−1, and zero-filled at invalid pixels. Deployment uses the onboard ZED2i and needs no external localization, object detector, state estimator, or runtime soccer planner.

Evaluation protocols and metrics

The same checkpoints, preprocessing, reset distribution, and thresholds are retained across each controlled comparison.

Metric definitions

Metric	Definition
SR	Shooting: the ball crosses the goal line in one kick. Dribbling: the robot stays upright and both robot and ball end within 1 m of the target; simulation also requires ordered waypoint visits.
r_contact	Fraction of shooting trials with valid foot–ball contact followed by goal-directed ball exit.
E_ang	Angular error between body heading and the robot-to-goal direction.
E_vel	Mean commanded-versus-actual ball-velocity error after engagement.
R_bib	Fraction of engaged dribbling time with the ball inside the controllable band.
B_turn	Absolute achieved-versus-commanded mean-turn difference divided by command magnitude.
RR_τ	Fraction of reaching rounds whose minimum ball–goal distance is below τ.

Shooting protocol

For free kicks, the robot starts 5 m in front of the goal while the ball spans x∈[0,4] m and y∈[−3,+3] m. For penalty kicks, the ball is 1.5 m in front of goal center and the robot starts 2–3.5 m behind it at −100° to +100°. Trial counts are balanced across spatial strata.

Shooting simulation and real-robot difficulty regions — Shooting configurations and difficulty regions used in simulation and on the robot.

Repeated-S slalom fixture

Field	Value / definition
Task grid	N∈{3,5,7} turns × α∈{60°,90°,120°}; 9 tasks and 900 trials
Waypoints	N+2 turning points + N+1 midline gates = 2N+3 total; arm length L=4.0 m
Commands	Waypoint attraction plus pole repulsion, snapped to 12 directions; k_att=1.0, k_rep=0.6, d₀=1.2 m
Engagement	Ball spawns 0.6 m ahead; first contact; 0.15 m/s speed threshold; 5 s timeout
Progress	0.75 m waypoint radius; 5 s segment timer refreshed at each waypoint
Control band	Robot–ball distance 0.42±0.10 m; ball speed 0.20–1.50 m/s; velocity not opposite the command
Failure	Fall, timeout, out-of-band dwell over 3.5 s, or robot–ball distance over 0.80 m

Repeated-S slalom trajectories across pole counts and turn angles — Successful trajectories are aligned at engagement; color encodes speed and the shaded region spans the 15th–85th percentiles.

Dribble Master protocol

Field	Dribble Master	DAVIS reproduction
Robot / input	Booster T1; detector ball position + in-view flag	Noetix E1; depth image
Command	Human-issued global ball-velocity vector	Scripted direction only; potential field for reaching, fixed heading for turning
Reaching	Distance unreported; final error <1 m	d∈{3,5,8} m; minimum distance below τ∈{0.5,1.0,1.5} m; end at 0.5 m or timeout
Obstacle	Physical, not perceived by policy	Virtual midpoint pole, not perceived by policy
Turning	±45°/90° on entry to a 0.4 m circle	±30°/45°/60°/90° after 1.1 m forward progress
Trials	5 rollouts per turn in simulation; 15 per real task	50 simulation rounds per condition; Wilson 95% CI

Dribble Master reaching and obstacle results

Paper Table VI reports target reaching and single-obstacle trials; Table VII separately reports turning. Time is measured from first ball contact until the ball reaches the 1 m tolerance, as mean ± SD over rounds that reached the 1 m tolerance; parentheses give the number of those rounds.

Task	d	RR_0.5	RR_1.0	RR_1.5	Time to 1 m
Target reaching	3 m	0.88 [0.76, 0.94]	0.88 [0.76, 0.94]	0.90 [0.79, 0.96]	3.0±0.6 s (44)
	5 m	0.82 [0.69, 0.90]	0.84 [0.71, 0.92]	0.84 [0.71, 0.92]	5.7±0.5 s (42)
	8 m	0.78 [0.65, 0.87]	0.80 [0.67, 0.89]	0.80 [0.67, 0.89]	9.4±1.1 s (40)
Single obstacle	3 m	0.92 [0.81, 0.97]	0.92 [0.81, 0.97]	0.92 [0.81, 0.97]	4.8±1.1 s (46)
	5 m	0.94 [0.84, 0.98]	0.96 [0.87, 0.99]	0.96 [0.87, 0.99]	6.4±1.1 s (48)
	8 m	0.94 [0.84, 0.98]	0.94 [0.84, 0.98]	0.94 [0.84, 0.98]	9.9±1.1 s (47)
DM reaching (real)	Unreported	—	0.87 [0.62, 0.96]	—	22.0 s
DM obstacle (real)	Unreported	—	0.93 [0.70, 0.99]	—	31.4 s

DAVIS rows are simulation results with n=50; the two reference rows are Dribble Master real-robot results with n=15 at an unreported distance.

Dribble Master turning results

DAVIS uses 50 simulation rounds per condition, with achieved turns summarized over the scored rounds below. Dribble Master turning values come from MuJoCo with 5 rollouts per condition.

Command	Scored	DAVIS achieved mean±SD	Bias	Per-round	DM mean	DM bias
30° L	40	+28.6±20.5°	4.8%	52.9%	—	—
30° R	39	−25.5±15.3°	15.1%	41.5%	—	—
45° L	44	+39.3±14.9°	12.6%	27.6%	+43.58°	3.16%
45° R	44	−44.8±18.2°	0.5%	32.1%	−46.44°	3.20%
60° L	43	+60.4±19.5°	0.6%	26.3%	—	—
60° R	41	−53.7±17.2°	10.6%	23.3%	—	—
90° L	42	+84.0±25.4°	6.7%	18.8%	+88.06°	2.16%
90° R	43	−84.9±22.6°	5.6%	19.7%	−87.98°	2.24%
Mean bias	—	—	7.1%	—	—	2.7%

Real-robot shooting targets

Difficulty	Distance	Target angles
Easy	2.5 m	−30°, −5°, +5°, +30°
Medium	3.0 m	−30°, −5°, +5°, +30°
Hard	2.5–3.0 m	−45°, +45°

All 12 positions face the goal. The ball starts 1.5 m in front of the goal; success requires the ball to cross the goal line completely in one kick. Real dribbling covers 4–5 m reaching at ±30°, obstacle traversal with 60–120° turns, and slalom through 2–4 targets spaced 2–3 m apart.

Real-robot multidirectional shooting sequence — Multi-directional shooting on the Noetix E1.

Foot and ball during a real-world dribbling trial — Foot–ball contact during real-world dribbling.

Observation, optimization, and auxiliary supervision

Privileged labels shape training; the deployed actor receives only onboard observations and predicted auxiliary features.

Observation and network configuration

Item	Setting
Actor sensory input	Single depth frame + five-step proprioceptive history; optional low-dimensional task command
Proprioception	81-D for shooting; 84-D for dribbling
Depth encoder / latent	LightDepthEncoder / 32-D
HIM hidden-state estimate	3-D
Actor / critic MLP	[512,256,128], ELU; normalization enabled
Auxiliary heads	Task-dependent: ball geometry and visibility; plus goal geometry and visibility for shooting
Training-only critic signals	Base linear velocity, foot contacts, terrain scan, object geometry, auxiliary labels, and curriculum variables

PPO and AMP-HIM settings

Item	Value
Rollout / optimization	24 steps per environment; 5 PPO epochs; 4 mini-batches
Learning rate	1×10⁻³, adaptive
γ / λ / PPO clip	0.99 / 0.95 / 0.2
Entropy / value loss / max gradient norm	0.01 / 1.0 / 1.0
Discriminator	Wasserstein; [1024,512]; lr 5×10⁻⁶; replay buffer 200,000
AMP reward coefficient / interpolation	0.8 / 0.8
Auxiliary loss weights	Goal geometry 1.0; ball geometry 1.0; goal visibility 0.2; ball visibility 0.2

The 136-D AMP state uses joint positions, key-body positions, base linear and angular velocity, joint velocity, and foot contacts. Head dimensions are masked so imitation does not constrain active visual tracking.

Visibility-aware auxiliary features

For each object i, geometry and visibility heads use the current depth latent. Coordinates are head-yaw aligned: x forward, y left, and z upward. An object is visible when any predefined structural point lies inside the valid camera field of view and depth range; the geometry target remains its center.

ŷ_{t}^{i} = h_{geo}^{i} (z_{t}^{d}), {v̂}_{t}^{i} = h_{vis}^{i} (z_{t}^{d}), p_{t}^{i} = σ ({v̂}_{t}^{i})

L_{geo} = \sum_{i} v_{t}^{i *} ℓ_{geo} (ŷ_{t}^{i}, y_{t}^{i *})

L_{vis} = \sum_{i} λ_{v} BCEWithLogits ({v̂}_{t}^{i}, v_{t}^{i *})

L_{aux} = L_{geo} + L_{vis}

Geometry is supervised only when the object is visible. The source leaves ℓ_geo generic; visibility uses binary cross-entropy on logits.

Ground-truth to prediction transition

β_{k} = \max (β_{k - 1}, \sqrt{clip (\frac{k - k_{0}}{k_{1} - k_{0}}, 0, 1)})

α_{k} = 1 - β_{k}

y_{t}^{e, i} = (1 - β_{k}) y_{t}^{i *} + β_{k} ŷ_{t}^{i}

p_{t}^{e, i} = (1 - β_{k}) v_{t}^{i *} + β_{k} p_{t}^{i}

c_{t}^{i} = clip (stopgrad (p_{t}^{e, i}), 0.05, 1)

a_{t}^{aux} = [c_{t}^{i} y_{t}^{e, i}, c_{t}^{i}] i

The square-root ramp is monotone and can pause when the auxiliary-loss EMA is too high. The detached gate prevents policy gradients from changing visibility confidence through the actor input. At deployment β=1, so every auxiliary feature comes from depth prediction.

Training loss with and without ground-truth annealing — Auxiliary-loss behavior with GT-to-prediction annealing and without annealing.

Task rewards

Shooting and dribbling retain the same observation-to-control interface while changing task objectives and reward terms.

Goal-directed shooting

Training uses 1,024 parallel environments, 20 s episodes, and a 0.005 s physics step. A near-ball sigmoid gate modulates alignment and contact rewards.

Reward term	Weight	Reward term	Weight
Approach ball	3.0	Adaptive velocity	2.0
Head track ball	3.0	Keep in horizon	−0.5
Body follow head	2.0	Robot behind ball	1.0
Lateral kick-line penalty	1.0	Head yaw align	1.5
Ankle toward ball	2.0	Ball velocity	4.0
Ball velocity direction	1.5	One-shot score	10.0
AMP motion prior	0.8	Safety / style	Task-dependent penalties

Directional dribbling

The policy follows a 12-way command ring for 60 s episodes. The ball-band half-width tightens from 0.15 to 0.10 m, and the out-of-band dwell limit tightens from 6.0 to 3.5 s.

Reward term	Weight / setting
Ball command	5.0
Body command	0.4
Dense re-approach	2.0
Out-of-band	−1.0
Ball in view	1.0
Body follow head	0.6
Fall / bad orientation / lost-control termination	−1.0
Head-masked Wasserstein AMP prior	Coefficient 0.8; interpolation 0.8
Safety / style	Task-dependent posture, sliding, force, limit, acceleration, action-change, and upper-body penalties

Active-vision ablation results — Active-vision ablation under the shared evaluation protocol.

Domain randomization and sim-to-real

Camera alignment and randomized sensing, dynamics, and disturbances are applied before ONNX deployment.

The simulator renders calibrated ZED2i distance-to-image-plane depth at 168×94, then crops it to 168×80. Both simulation and robot use the 0.3–5.0 m range, 2D/5.0−1 normalization, and zero for invalid pixels. Temporal context comes from proprioceptive history rather than stacked depth frames.

Category	Perturbation	Range / value
Depth	Valid depth	0.3–5.0 m
	Distance-proportional / Gaussian noise	0.01 / 0.005
	Salt-and-pepper / stripe dropout / edge drag	0.01 / 0.005 / 0.01 probability
	Rectangular mask	0.1 probability; 10–25% size
	Ball-region occlusion	0.2 probability; 2–10 px radius
	Partial occlusion	45–85%
	Border near-field noise	0.15 probability
	Camera intrinsics	±0.3–±0.5 px
Camera pose	Translation	0.02–0.04 m
Camera pose	Rotation	0.025–0.060 rad
Robot	Joint-position offset	±0.02 rad
	Base COM	±0.05 m
	Base mass	±5 kg
	Actuator stiffness / damping	0.8–1.2 scale
Ball / pushes	Ball mass	0.36–0.56 kg
	Push interval	3–15 s
	External linear velocity	x,y∈[−1,1] m/s; z∈[−0.2,0.2] m/s
	External angular velocity	[−0.78,0.78] rad/s

Additional skills

The same depth-only actor interface is retrained with task-specific objects, rewards, resets, and schedules.

Powerful shot

A near-ball gate first weights foot-speed effort by distance, then introduces horizontal ball-speed and displacement rewards. The scales enter sequentially; the AMP kick prior starts once enough environments reach the near-ball zone. Deployment still uses the depth actor without a separate locator or shooting controller.

Obstacle-aware dribbling

The existing per-object mechanism is extended with an indexed obstacle geometry/visibility head on the shared depth latent. During training, direction-aware rewards track a command deflected tangentially around the nearest in-path obstacle. Outside its influence radius, the reward target returns to the requested direction:

{d̃}_{t}^{cmd} = \frac{d_{t}^{cmd} + κ w_{t} a_{t} t_{t}}{‖ d_{t}^{cmd} + κ w_{t} a_{t} t_{t} ‖}

a_{t} = \max (u_{t} \cdot d_{t}^{cmd}, 0), t_{t} \cdot u_{t} \leq 0

Here u_t is the ball-to-obstacle bearing, t_t is the command-normal unit vector toward the obstacle-clearing side, w_t ∈ [0,1] is a proximity weight that vanishes beyond the influence radius, and κ is the steering gain. Every direction-aware dribbling reward tracks the deflected command. The result is obstacle avoidance within the same depth-only actor and control loop, without a separate runtime avoidance controller.

Auxiliary losses for ball and obstacle prediction — Ball and obstacle auxiliary heads converge during training.

Ball-loss recovery

0.5 scontinuous invisibility before search penalty

0.2search-penalty weight

≈0.5 m/sforward-speed limit while invisible

0.6invisible-speed penalty weight

12k + 36khead reward start + ramp

48k + 48kbody-follow start + ramp

The policy searches and slows down through learned rewards and the visibility timer; no separate search controller is introduced.

BibTeX

@misc{jin2026davisdepthonlyendtoendactivevision,
  title={DAVIS: A Depth-Only End-to-End Active-Vision Framework for Humanoid Soccer Skills},
  author={Jiakang Jin and Yixiao Huo and Pengyuan Wang and Yinan Han and
          Tingxuan Zhang and Zhuobing Zhao and Xuanxin Zhou and Zhangchen Ye and
          Enxuan Ruan and Yifei Bao and Jiankun Yang and Chenghao Sun and
          Wenhao Cui and Xiaoyu Tian and Yiming Li},
  year={2026},
  eprint={2609.28175},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2609.28175}
}