These 7 commits are when the Protocol Buffers files have changed:
| Commit: | f724f3b | |
|---|---|---|
| Author: | Chris Paxton | |
| Committer: | GitHub | |
[Major update] Spot support, open-world exploration, instance mapping fixes, and RRT for pick and place in cluttered scenes (#435) * Fixed the timestamp format * do not publish for now; update UI code * add some documetnation or something idk * add some more documentation i guess * handle limited obs * finish everything * add some comments * Added the replacement of Instance id: with image in chat UI * add rpc * update demo * update to not rewrite images * added modals and new layout * did some future-proofing for modals when there are multiple crops * demo chat class component * time to remove hardocding * install updates * hide bullet * Sasha's changes on the laptop * poll_delay * publish limited obs * limited obs working * add utils * Added checks for fields tha are not available. * WORKING DEMO NIGHT BEFORE V0 * WORKING * dd some comment and switch to trace * update configuration + some minor debugging + update motion planner setup * update configuration for exploring * added aws ip address comment * update configuration and config * remove start streaming * update config - motion planning setup * remove start streaming * quick status update * Improved the text of user reqeust * param flag to finish before sweeping * move arm to a custom pose * added find_only * added find_only * sleep in * fix comment * switch from 90 to 75 * self missing * Added fix to view modal instance * refactor * cleanup * refactoring and remove publish_obs * update habitat lab * update habitat-lab * update habitat-lab to main * update * update ovmm * Removing GOAT code * removing GOAT stuff * Moving OvmmPerception to its own folder * refactoring the perception wrapper code * remove mock agent * refactor navigation client code (tools for move and place) * remove unused code * refactor * fix some issues with ik * refactored ui to use class components * fixing * refactored ui to use class components * rm vlm-planning related code * remove GOAT stuff - that can be a separate release * binit's suggested changes from the readme * update config + launch file * delete vlm_demo from real world oovmm * update grasping and refactor some things * fix bug * tweak setup and make sure we are getting semantic sensor from env * fix some issues to make it easier to just pass in a name into grasping * update grasping and make it work * remove minigpt4 submodules * clean up docs and make sure place computation works. Next step: generate path to this point * remove goat docs * improve placement code * remove minigpt4 folders * added modalities and instance 2d seg to scannet datasets * fix broken import * added modalities and instance 2d seg to scannet datasets * patch for torch 1.x indexing * fix a few scannet eval hydra configs * update spot installation * revert GOAT planner, get ovmm sim running * update zerodepth test script * add comment to script * update and break out the parameters to make it a bit easier * add parameters file to make this easier * update demo data script * Instance ids and saving data * update readme and documentation * update agent * update isort * move spot demo agent into a new file -- not tested * isort fixed * update parameters * clean up spot ipython notebook * update configs for refactor * update env and clean up other code * libboost in circleci * install deps for boost * add conda install for pin (optional) * update install with zerodepth * fix control mode; delete more Spot stuff * add atomicwrites + mamba->conda * ok keep mamba because conda is so slow * mamba -> conda and see if circleCI still works * fixes from the lab * added refexp * fixes from the lab * typos * number of episodes not found * add reset function * execute trajectory is cleaner, improving setup, add loguru * add rosnumpy * update instructions, add deprecation * update real-world ovvm scripts to make sure they work properly * remove visualization flag * remove the map rotation inside semantic module * keep obstacles in frontier * remove some code; make sure heuristic policy runs * remove import statement * remove viz flag * default config path * add extract scalars function * remove unused languagenav_agent * imports * remove dependency for sep repo * match explored radius to main * set goal ids correctly in sim * remove goat languagenav files * add docstring * remove hab_info dep * add config * typo * spot code fixes * remove minigpt4 directory * fix linter issues * remove some deps * Ovmm sim demo integration (#439) * added emacs gitignore * first pass at ovmm sim * able to run frontier exploration * added make video to see what's happening * commented out some unused code * added task goals and attempts at grasping * fixing installation issues for simulation + make sure rendering test works * update script * fix some install issues and add habsim * Update install stuff, fix issue with pillow version * dd test script * save computed point cloud * save some more data * add voxel code * Script to read information from voxel map and remove unused import in __init__ * fixing some issues * tart exploring again * try-except to make sure we save data even if things are incorrec * try rotating in place; add support for this * move client and fix inputs * trying to get the robot setup * rotating in place * moving around and exploring * modify agent to improve exploration; still seems to get stuck * nter * linter * first attempt at execute vlm plan --------- Co-authored-by: Chris Paxton <cpaxton@jhu.edu> Co-authored-by: Chris Paxton <cpaxton@fb.com> * update the agent code to fix linter problems * Replace conda with mamba in install_deps.sh * Conceptfusion (post-demo) (#441) * add scannet dataset creation * moved scannet to home_robot/datasets and added licenses * adding init.py * Add referring expressions to scannet dataset * SparseVoxelMap and InstanceMemory use tensors and work on gpu * added instance visualization and some misc documentation * nbstriput to not commit notebook outputs * nbstriput to not commit notebook outputs * remove gitattributes -- either we can use git or pre-commit. dont need both. * strip output for circleci * add metadata strip * add jupyter nbconvert for metadata strip * return None for RGB cluster when no RGB provided * return None for RGB cluster when no RGB provided * strip output from nb * Make changes to installation to (hopefully) make build process easier (#351) * strip output from nb * add object detection eval helpers for scannet * scannet build_3d_map and eval example * scannet build_3d_map and eval example * referring expression evals and grounded sam on scannet * added concept fusion demo * adding eval script for full scannet * handle nans and small bounding boxes * add instance association within category * added bbox postprocessing NMS and box-tightening * removed extra code * Add the DBScan clustering * new project, moved major functionality to conceptfusion * added visualizations, tuned parameters * incorporate changes from Sasha * Try Sasha's idea for quering * added config * semmap updatae * float instead of double * cleaning up merge * added a little documentation * added a litle documentation --------- Co-authored-by: Sasha Sax <asax@stanford.edu> Co-authored-by: Chris Paxton <cpaxton@fb.com> Co-authored-by: Karmesh Yadav <ykarmesh@gmail.com> * add network option? * command line arguments for install script * remove centernet * update robot model * minor changes to set up code * remove hardcoded keys * update challenge.md * add keys in hssd_demo.yaml * fix instructions for docker build * rviz image bug fix * example * minor bug fixes and helpful logging * delete logger file -- not needed * testing rpc stub * updates to ovmm sim * task_rpc_env_pb2_grpc should be imported after src.rpc is imported * grasping fixes * new fixes * new fixes * Update challenge.md * Update challenge.md * minor fixes + new timestamp for datadump * fixes for sim * camera pose fix + refactor * update hab-lab commit * framework for sample strategy * slight fix * . * remove print * make habitat camera pose work for SVM in sim * Update spot.md * Update spot.md * Update spot.md * add clip queries to svm * minor tweaks for gpt4v agent * fixes for evaluations * adding nr3d only * adding nr3d only * optimize image saving * make padding a param * improving some parameters in stretch demo code * add in-place rotation for stretch demo * rotations work in home * update robot demo code for exploration * fix some issues with Stretch script * formatting * update visualizer * update this code so that we can see whats going on * evaluator * linter errors (isort) * delete and add h5py version * dependency fixes * cleanup deps * roll back hssd-hab commit change; keep using ovmm branch with navmeshes * updating robot code * fixing simple heuristic example * switch to opengl camera pose via config, remove unused code * rename opengl to opencv * clean up debug statements --------- Co-authored-by: Oleksandr Maksymets <maksymets@gmail.com> Co-authored-by: Sasha Sax <asax@stanford.edu> Co-authored-by: PrieureDeSion <shah@cs.berkeley.edu> Co-authored-by: Jay Vakil <61257499+Jdvakil@users.noreply.github.com> Co-authored-by: pmcvay <pmcvay@meta.com> Co-authored-by: keke-220 <xzhan244@binghamton.edu> Co-authored-by: Sriram Yenamandra <yvsriram99@gmail.com> Co-authored-by: jdvakil <jdvakil@gmail.com> Co-authored-by: Sasha Sax <sax@berkeley.edu> Co-authored-by: Karmesh Yadav <ykarmesh@gmail.com>
The documentation is generated from this commit.
| Commit: | c935bf2 | |
|---|---|---|
| Author: | sneha | |
| Committer: | sneha | |
moved from scannet, cleaned up
| Commit: | 446ffc5 | |
|---|---|---|
| Author: | sneha | |
working PlanHighLevelAction, that accepts 10 object crops
| Commit: | 39002b8 | |
|---|---|---|
| Author: | sneha | |
wip - send instance images from spot obs pickle, receive action
| Commit: | d349cc4 | |
|---|---|---|
| Author: | sneha | |
| Committer: | sneha | |
load llama model and respond to question
The documentation is generated from this commit.
| Commit: | f287064 | |
|---|---|---|
| Author: | sneha | |
more fields
| Commit: | 847c2ef | |
|---|---|---|
| Author: | sneha | |
| Committer: | sneha | |
send image and rgb