Skip to main content

October

Thursday 1st​

  • Cleaning up September notes
  • Investigating monopulse radar (https://en.wikipedia.org/wiki/Monopulse_radar) for improving angular accuracy
  • Added system for calculating if tracks have line-of-sight of the origin (i.e. the sun)
    • Use this for optical tracking, calculating reflected light
    • Building a physically based optical sensor model
  • Jobbified optical scan system (500us -> 25us)

Friday 2nd​

  • Discord post
  • Converting kalman state propagation to a job query (400us -> 35us)
  • Experimenting with NEES auto tuning, picking kalman process noise

Monday 5th​

  • Calculating NEES in sensor system for position variance
  • Auto tuning Q (process noise) in sensor system based on NEES
  • Building RADAR tracking system for Kalman
    • Counting tracking targets per-sensor
    • Doing the actual tracking
  • Moving new sensor system from prototype folder into main project, deleting old sensor system
    • Deleting:
      • RADAR noise jammer
      • V2 track init
      • Many tests covering old sensor system
    • Fixing tests
  • Lots of code movement, cleaning up sensor system file organisation

Tuesday 6th​

  • Unit testing
    • NativeArray2D
      • Disposal doesn't work properly. Removed it, this is a wrapper struct that does not take ownership.
    • PhasedChunkQuery
    • CommandBuffer.DelayedResolve
    • QuaternionDouble
    • RigidTransformDouble
    • MathEx
    • JobTextureWriter/Colormap
  • Started sketching out types for Kalman propagation
    • Cold/Live/Attached
    • Updated Kalman propagation to only run for Live tracks

Wednesday 7th​

  • Modify kalman propagate to handle "live" tracks
    • Adding helper to Kalman lib to build control matrix
    • Propagating with kepler forces
      • Measured from estimated position
      • Assumed to be constant acceleration over timestep
  • Modify kalman propagate to handle "attached" tracks
    • Skipping this for now, let's see if we can get away with just 2 track types (cold and live)
  • Investigating weird tracks which are "detached" from true state in test scene
    • Drawing lines where the track and the true state are very far apart
    • Not a real problem - caused by the balls bouncing which is not modelled by the Kalman process model
  • Modifying adaptive step size to increase process noise faster than decrease
  • Testing Kalman sensors in orbital scene
    • Modifying test detectable objects to be NBodies
    • Investigating why live integration isn't working
      • Bug with ECS query
    • Testing live vs rail integration
      • Both seem fine
    • NEES values get very elevated at high time speeds
      • Probably to be expected, sensors only run once per frame regardless of time speed so there are a lot more errors to accumulate
      • Clamping Q value prevents it exploding during high time speed
        • Might want to consider adapting Q differently based on time speed, e.g. adapt more quickly when DT is first reduced
    • Spawning objects in something like actual orbits
    • Investigating LoS checks
      • Not a real issue - Test planets had zero size
    • Quality of track drops very quickly when there is no line of sight!
      • Due to high time speeds?
      • Due Larger time step errors?

Thursday 8th​

  • Investigating divergence at high time speeds
    • Obviously caused by errors in large time steps. Switching Kalman to use a constant jerk model, instead of constant acceleration.
    • Making Q adaptive run in real time instead of sim time, so high time speeds don't make it explode
    • Changing time speed a lot (1000x to 1x) causes state to drift
      • Possible there's something not time-invariant, causing scale drift
      • Or it could just be because there are intrinsically larger errors at high time speed
    • Building a system to propagate "attached" tracks
      • For now very much cheating, just injecting true position as a very confident sensor before propagation.
      • NEES value for this near perfect track is still high!?
      • Moving orbital systems before sensor systems seems to have fixed this - ordering issue with position updates:
        • Orbital systems set position to current timestamp
        • Sensor systems propagate to current timestep and calculate error (NEES)
        • If this order is inverted the NEES calculates on a newly propagated position, but a stale actual position

Friday 9th​

  • Testing fixed sensor system
    • Q (process noise) is hitting the minimum constantly (this is good)
    • It converges over time, even with very noisy sensors
    • It explodes if the noise is set too high!
  • Tightening up optical physics sim
    • Better SNR scaling (storing signal and noise and scaling them separately)
    • Added a soft floor to angular accuracy
    • Extending optical sensor struct
      • Sensor size
      • Focal length
      • Using this to calculate time required to scan entire sky, sensor return is divided by scan time

Monday 12th​

  • Fixing polyline error (attempting to render lines with < 2 points)
  • Investigating GunLeadingOrbital scene to check if it still works
  • Create a system to designate tracks as cold/live
    • Spawn some really far out tracks, coming towards Earth (start cold, become live)
    • Spawn some tracks near Earth, but very high speed (start live, become cold)
  • Creating components to define perimeters
  • Creating tag components to mark tracks
  • Creating system to classify tracks
    • Classify in job
    • Extract results
    • Apply on main thread
  • Fixed bug with FindMinIndex (mutating input array)
  • Sanity checking optical physics again
  • Investigating possible bug with Myriad.ECS CommandBuffer not properly removing tag components
    • False alarm
  • Perimeter system test coverage

Tuesday 13th​

  • Fixing HandyJobs namespace
  • Using HandyJobs instead of jobs in main project (removing duplicated code)
  • Working on system to manage tracks, switching between "Cold" and "Live"
    • Using ParallelAppendBuffer (see below)
    • This system is actually fairly expensive (130us), is it worth doing when track propagation is actually very cheap?
      • We could drop the entire hot/cold concept, which removes a lot of complexity
      • Removed it
  • Geometry code review
    • Fixed IntersectsRay bug
    • Fixed BoundsCone.Contains bug
    • Removed Union for bounds volumes (buggy, not used)
    • More robust handling of ClosestPoint for parallel rays
  • Created ParallelAppendBuffer for the commonly occurring pattern of accumulating per-thread result lists
    • Unit testing
    • Retrofitting into perimeter system from yesterday
  • Deleted Octree, it's unused now and the Sweep&Prune does the job better
  • Removed unused BitList256
  • Fixed potential job safety issue with sweep and prune (never blocking on handles)
    • Not actually a safety issue today, the handles are always completed. But it did mean we had an ever growing list of completed handles!
  • Fixed bug in sweep and prune which could return stale results in collision queries.

Wednesday 14th​

  • Refactoring spawning in GunLeadingOrbital scene so I can add sensors
  • Adding helper functions for initialising entities as sensor platforms
  • Set up sensors for gunnery defender
  • Updating range calculation system to tag tracks as "hot" (getting closer) or "cold" (getting further away)
    • Fixed ParallelAppendBuffer when used across more than 1 chunk (disabling safety checks)
  • Improved collision checks for broadphase objects broadphase (proper velocity sweeping)

Thursday 15th​

  • Fixing edge cases in PID (negative DT breaks state)
  • Fixing edge case in Exponential Moving Average (negative smoothing factor)
  • Tweaking bullet rendering - less fade at extreme thin scaling factors. Bullets are visible from further away
  • Fixed SNR calculation for RADARs affected by band limited noise sources (e.g. active jammers)
  • Fixed reference SNR for optical sensors (off by a factor of PI)
  • Reviewing bullet link culling
    • Could be improved by slowly "fading" out links instead of deleting them. Just remove bullets below the density limit until zero bullets remain.
    • Implemented by clearing the lowest bit. This is efficient but visually biased since the bits are visually ordered.
    • Randomised the bit to be cleared.

Friday 16th​

  • Added TurretRotationError which stores the angular error between turret and target
  • Adding InPlaceList8 and InPlaceList4 in addition to the already existing InPlaceList16
  • Creating GunTriggerSystem which sets the trigger when the angular error is low enough
  • Created system to accumulate turret errors
  • Fixing default component inspector handling for read-only properties
  • Creating system to control trigger based on policy
    • Simple angle error < threshold policy for now
  • Testing
    • Added some spin to test platform
      • Missing lots!
    • Guns occasionally shoot bullets at the moon

Monday 19th​

  • Investigating missing when ship is spinning
    • Possible system ordering issue - ship turns, transform tree is calculated, then the gun turret aims. Aiming is always 1 frame behind?
      • Problem: Transform must run before (updated WorldTransform is needed for aiming) and it must run after (updated WorldTransform is needed for gun firing)
      • Hack: Add another copy of the hierarchical transform system. This fixes it!
      • Adding the ability to filter the hierarchical transform system
        • Exposed filtering in Myriad/Unity integration package
      • Switched out hierarchical transform in turret system group to a filtered one, only updating turret components. it works!
  • Fixed an edge case in bullet rendering - no rendering metadata available if the gun that fired a bullet is destroyed. Switched to using the metadata which is attached to the bullet itself.
  • Fixed an edge case in intercept calculation - projectile and target have identical speeds.
  • Added QueryBuilder.Clone to Myriad.ECS
  • Considering flak bullets
    • Experimenting with GPU driven rendering of bullets, adapting current shader to switch from rays to a growing billboard
    • Playing frames of a sprite sheet on billboard
  • Fixed turret error value resetting to zero prematurely
  • Renormalising turret rotations
  • Redesigning turret solving system

Tuesday 20th​

  • Setting up test scene for new turret aiming system
  • Adding Collect type queries to Myriad.ECS for EntityId instead of Entity (works better with Unity jobs)
  • Building new IK system for turrets
  • Todo:
    • Calculate errors for turret components
    • Remove old error accumulation (no longer needed)
    • Parallelise this system if possible
      • Parallelised error calculation
      • Parallelised IK solving
    • Port into main and replace current IK system
    • Investigate if GetComponentRef is a parallel safety leak

Wednesday 21st​

  • More parallelisation of IK system (removing the required blocking)
    • Investigating potential job handle leak in parallel query
      • Input handle is lost when the query matches no entities! This breaks the job dependency chain.
    • Removed the final blocker, parallelising the entire system
  • Tightening up bounds on angle-error measurement
  • Moved system into main project, removed/replaced old systems
  • Investigating potential safety leak in GetComponentRef
    • Single component variant is ok
    • Multiple components variants are unguarded!
    • Fixed by adding appropriate Block calls into Myriad.ECS
    • Updated in main project
  • Adding larger salvos of simultaneous shots to orbital defence scene, this will push PDC system harder
  • Debugging issue with guns sometimes firing off to the side, even though they have a valid target
    • Seems to be due to maximum intercept time going very high. Added a maximum time limit
  • Fixing a couple of minor bugs in EventBus
    • Add/Remove listener in response to an event breaks
    • Better handling of re-entrant events (sending in response to a send), queueing them up and sending them after
    • Better handling of exceptions in events (collecting all exceptions into one AggregateException)

Thursday 22nd​

  • Code review of extensions methods
    • Removed duplicated code in ArrayExtensions
    • More robust handling of invalid inputs in BoundsExtensions.FromMinMax
    • More test coverage on BoundsMinMax
    • Removed dead code in CameraExtensions
    • Fixed comment in ComputeShaderExtensions regarding the range of soft-doubles
    • More input validation in ComputeBufferWriteHandle
    • Normalising return from double3.Perpendicular, consistent with float3
    • More coverage for GameObjectExtensions
    • Proper handling of negative values in Kilogram string formatting
    • Better handling of underflow in NativeHashMap.DecrementKey
      • Remove Increment/Decrement methods. They were used in Octree which was removed the other day.
    • Improved copying of blittable values
    • More input validation in DoubleExtensions
    • More input validation on SpanExtensions, particularly unsafe reinterpreting spans
    • Fixed out of bounds indexing in v128.GetBit

Friday 23rd​

  • Improving Myriad.ECS CommandBuffer by reducing the amount of blocking it does on parallel work.
    • Fine grained blocking in ApplyStructuralChanges
    • Fine grained blocking in CreateBufferedEntities
    • Per-entity blocking in RelationshipBinder
      • Narrowed down to just the specific relational component type being touched, which should never be in a job anyway
    • Imported into main project.
      • This has helped a bit. More time is being spent in the explicit job gates instead of the command buffer system.
  • Tightening up attack patterns in gunnery tests scene
  • Ensuring more jobs are using Burst compilation
  • Beginning design of self defence system

Monday 26th​

  • Switching projectiles (test projectiles plus bullet simulation points) to live integration instead of rail integration, faster to setup and teardown
  • Adding inspector for BulletChainGroup to help pick bullet culling settings
  • Creating defence platform components
  • Creating defence platform systems
    • Gather platforms (hosting auction)
    • Gather weapons (bidding on target)
    • Gather targets from sensor tracks (bid items)
    • Run auction
    • Add helpers to setup entities
      • Setup platform
      • Setup weapon
  • Adding JobHandle disposal to ParallelAppendBuffer

Tuesday 27th​

  • Extracting targets to bid on
  • Adding a synchronous overload to AuctionAlgorithm, so it can be used inside a job
  • Adding helper to extract list of values from ParallelMultiHashMap
  • Calculating values for jobs in auction
    • Calculate base score
    • Calculate per-weapon modifier
  • Extracting assignments
    • Per-auction extraction
    • Extraction back into ECS
      • Need a component to store target & system to aim at tracks, currently just a placeholder that matches no entities

Wednesday 28th​

  • Created a component to store weapon target assignment
    • Need a system that computes targeting from this assignment
  • Testing out prototype pipeline (doesn't actually do any work yet, just checking dependencies and dataflow)
    • SortJob not properly depended on
    • Switching slices list to a NativeHashMap, keyed by platform ID
    • Fixed parallel access to NativeList
    • Properly sizing the bid value list
    • Early out if there are no weapons (bidders) or targets to bid on
  • Designing "threat score" function
    • Threat(Track):
      • Impact probability - chance it hits at all
      • Impact severity - how bad would a hit be
      • Urgency - how soon will it hit
    • Bid(weapon, track):
      • Track threat score
      • Probability of neutralisation by this weapon
      • Cost of using this weapon
        • Ammo/Power
        • Opportunity cost
  • Filtering tracks which are considered
    • Must be hot (closing range)
    • Must be within the outermost defence perimeter
  • Implementing base threat score
  • Adding matmul(doubleSymmetric3x3, double3) method

Thursday 29th​

  • Continuing implementation on Threat(Track) function
  • Adding IncludeScheduledQuery extension for single parameter queries
  • Tracking down a leak
  • Index out of bounds access
  • Splitting threat score calculation out into a separate system
    • Clear all scores
    • Copy data from platform to track (e.g. threat parameters)
    • Calculate new threat scores
    • This doesn't work very well, need various other resources that are already gathered in the threat pipeline that are painful to break out.
  • Added an overload for Kalman predict which doesn't need Q matrix
    • Using this in track threat assessment

Friday 30th​

  • Writing threat assessment values back to ECS
    • Useful for debugging and possibly showing to player
  • Fixing edge case potential catastrophic cancellation in quadratic RootFinder
  • Unit tests for threat assessment function
  • Unit tests for GameObjectCameraFocusStack
    • Fixed bug with Trim losing all items
  • Fixed BaseVisualEffectPool retaining and returning dead items
  • Fixed SimpleRandom when initialised with seed=0