Proto commits in mondoohq/cnquery

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2025-05-05

⭐️ `lr`: add provider dependencies to schema (#5495) * ⭐️ `lr`: add provider dependencies to schema Resources within a provider can depend on resources from another provider, for example; The `os` provider depends on `network` provider. The `lr` parser already reads these imports but it wasn't passing it to the schema. This change adds the list of dependencies so that we can install them at runtime. Signed-off-by: Salim Afiune Maya <afiune@mondoo.com> * ✨ test dependencies in lr files Signed-off-by: Salim Afiune Maya <afiune@mondoo.com> --------- Signed-off-by: Salim Afiune Maya <afiune@mondoo.com>

The documentation is generated from this commit.

2025-05-02

⭐ v12 Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2025-05-01

⭐️ `lr`: add provider dependencies to schema Resources within a provider can depend on resources from another provider, for example; The `os` provider depends on `network` provider. The `lr` parser already reads these imports but it wasn't passing it to the schema. This change adds the list of dependencies so that we can install them at runtime. Signed-off-by: Salim Afiune Maya <afiune@mondoo.com>

The documentation is generated from this commit.

wip Signed-off-by: Salim Afiune Maya <afiune@mondoo.com>

The documentation is generated from this commit.

2025-04-22

⭐ v12 Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

✨ Add WIF auth support (#5439) * Add WIF token exchange calls to client * Add token fetching and handle exchange * Add function to figures out which provider to use * Add updated proto for upstream changes * Add ranger.go file for changed upstream.proto * Fix indentation * Add support for WIF config files * remove the default issuerUri from config.go, it's handled elsewhere * Use afero and add unit tests * Cleanup from when this was using flags * Clean up old and redundant code * Fix up tests

2025-04-21

Fix indentation

Add WIF token exchange calls to client

2025-04-17

✨ add new exception types

2025-04-16

✨ add new exception types

⭐ v12 Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2025-04-15

Fix indentation

Add WIF token exchange calls to client

2025-04-03

Add status to sbom package

2025-03-08

✨ ip.cidr, ip.address and raw IP storage (#5292) * ✨ ip.cidr and raw IP storage Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 rename mask -> prefixlength Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 expand int2ip types Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🐛 print CIDR of IP by default Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🐎 more efficient bitmask generator Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 📃 clarify IP internal vars Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * ✨ add ip.address Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🔨 switch from manual to proto-based IP marshaling Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 remove those unnecessary types Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 split up test for data conversion Signed-off-by: Dominik Richter <dominik.richter@gmail.com> --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2025-02-26

🐛 fix core `asset.labels` + ✨ new `asset.platformMetadata` (#5227) The labels of an asset come from the asset itself, not the platform, but as some point we made a mistake to use the platform labels, this PR is the first step to fix this issue. In this PR we are: * Introducing a new field to the proto message `asset.Platform.Metadata` with the goal to give more clarity and avoid confusions with `asset.Labels` * We are deprecating `asset.Platform.Labels` in favor of `asset.Platform.Metadata` (this field will be remove in `v12`) * We are introducing a new `asset.platformMetadata` to the `asset` MQL schema, this new field will have the `asset.Platform.Metadata` * We are fixing the `asset.labels` MQL resource to have the actual asset labels but, we are going to merge this field with the platform labels for backwards compatibility After merging this PR, we will start migrating the use of `asset.Platform.Labels` to `asset.Platform.Metadata`, so that in `v12` we can remove `asset.Platform.Labels. --------- Signed-off-by: Salim Afiune Maya <afiune@mondoo.com>

2025-02-22

wip: remove platform.labels for testing Signed-off-by: Salim Afiune Maya <afiune@mondoo.com>

2025-02-21

⚙️ expose platform labels as `asset.platformMetadata` Signed-off-by: Salim Afiune Maya <afiune@mondoo.com>

2025-02-07

✨ resource context in LR definition (#5176) Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2025-01-10

⭐️ Add tags to objectref (#5083) * ⭐️ Add tags to objectref We can use the tags to provide additional metadata for variants like what icon to use and what title to show An example: ``` queries: - uid: mondoo-edr-policy-ensure-edr-agent-is-installed title: Ensure EDR Agent is installed variants: - uid: mondoo-edr-policy-ensure-edr-agent-is-installed-macos tags: mondoo.com/filter-title: macos - uid: mondoo-edr-policy-ensure-edr-agent-is-installed-linux tags: mondoo.com/filter-title: linux - uid: mondoo-edr-policy-ensure-edr-agent-is-installed-windows tags: mondoo.com/filter-title: windows ``` * update example

2024-12-30

🧹 update documentation for protobuf and regenerate with latest protoc (#5045) * 🧹 update documentation of proto file * 🧹 update generated protobuf code

2024-12-03

Add disabled scoring system (#4944) Needed for https://github.com/mondoohq/cnspec/pull/1502

2024-11-11

⭐️ add more optional metadata fields to remediations (#4826)

2024-08-11

⭐️ product.releaseCycle resource (#4512)

2024-07-31

Capitalize Mondoo Platform in a few places (#4464) * Capitalize Mondoo Platform in a few places This also updates the short description of the platform that is used in `cnspec scan/shell help` ``` ... mondoo Scan the Mondoo Platform ... ``` Signed-off-by: Tim Smith <tsmith84@gmail.com> * Update Makefile Co-authored-by: Letha <letha@mondoo.com> * Update providers/mondoo/config/config.go Co-authored-by: Letha <letha@mondoo.com> * Update providers-sdk/v1/upstream/upstream.proto Co-authored-by: Letha <letha@mondoo.com> --------- Signed-off-by: Tim Smith <tsmith84@gmail.com> Co-authored-by: Letha <letha@mondoo.com>

✨ ListResources for resources explorer (#4448) Slowly starting to extend the functionality to cover exploration use-cases Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2024-07-24

✨ Add labels to the reporter assets. (#4401) Signed-off-by: Preslav <preslav@mondoo.com>

2024-07-23

✨ Add Azure ID and Azure sub external IDs to the sbom proto. (#4348) Signed-off-by: Preslav <preslav@mondoo.com>

2024-07-18

✨ Expose package vendor via MQL (#4357) Based on #7524 Signed-off-by: Christian Zunker <christian@mondoo.com>

2024-05-13

🧹 Drop deprecated v8 kind. Signed-off-by: Preslav <preslav@mondoo.com>

2024-04-18

⭐️ support env variables as credential type for inventories (#3792) This is especially useful when used in CI/CD environments. Users have a pre-defined inventory file with one or multiple entries but do not want to store the secrets in there. Instead the CI/CD run defines the secret via an environment variable. ```yaml spec: assets: - connections: - type: slack credentials: - type: env env: CUSTOM_SLACK_TOKEN ```

2024-04-17

🎉 v11 🎉 (#3755) * 🎉 v11 🎉 Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🟢 lint Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🟢 fix mockprovider (version bumped to v11) Signed-off-by: Dominik Richter <dominik.richter@gmail.com> --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2024-04-15

✨Add trace-id to sbom proto. (#3752) Signed-off-by: Preslav <preslav@mondoo.com>

⭐️ trace-id argument for cnquery (#3749) * ⭐️ trace-id argument for cnquery * 🧹 update tests

2024-04-04

⭐️ define proto for cnspec report (#3697) * ⭐️ define proto for cnspec report * 🧹 use new proto report format for sbom

2024-03-28

⭐ resources explorer API (#3668) * ⭐ upstream recording support Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🐛 ensure recordings get updated MRNs after discovery We previously only store the internal asset ID in recordings. However, whenever available, we want to use the MRN in the ID field. This will provide better lookups for the recording when loading assets via MRN. Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧪 Migrate Resources Explorer to cnquery Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 entirely restructure the resources explorer Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🐛 pull platform IDs into recording Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🟢 go mod tidy Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * ✨ mark upstream as hidden for now Signed-off-by: Dominik Richter <dominik.richter@gmail.com> --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2024-03-13

⭐️ new vuln mgmt proto api

🧹 support origin package information sbom (#3558)

⭐ add TechnologyUrl to platform + AssetUrlTrees to provider (#3540) * ⭐ add TechnologyUrl to platform + AssetUrlTrees to provider 1. Providers are extended to define their AssetUrlTrees. These are part of the config and can be loaded statically. This allows developers to define and extend existing AssetUrlTrees. Note that we currently maintain a `/technology=X` asset tree. We are in the process of adding all providers to this grouping strategy. 2. Extend the inventory definition of the asset `Platform` to include the TechnologyUrl. While assets may belong to a multitude of URLs, the platform definition only focuses on the technology URL. 3. Implement both for the `os` provider. Note: Initially this was designed to define the entire tree of URLs, including all OS names and versions. I noticed, however, that the structure of the tree doesn't change by different OS families, names, or versions, at all. Thus, we use `*` to allow for arbitrary terms, as these terms won't have any impact on the remaining URL. Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 check against platform name or version being empty Signed-off-by: Dominik Richter <dominik.richter@gmail.com> --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2024-03-12

⭐️ support scope mrn reading from service account (#3146)

2024-03-08

⭐ introduce asset URLs (#3529) * ⭐ introduce asset URLs From the code docs: Assets are generally structured in a giant graph. However, we often find it difficult to reason with arbitrary graphs. As humans, we tend to group assets into hierarchical tree structures, that make it easy for us to put them into a box and reason about them. For example: A techology-centric view of the world would group a VM in a cloud environment like this: /aws/accountX/ec2/instances/linux/debian/8.0 Every entry in this path structure follows a strict schema. Thus "aws" above is the chosen path value for the key "technology". As you can see, some keys lead to predefined (limited) values (technology can be aws, azure, os, k8s, etc), while other keys can have (almost) arbitrary values (eg account). Providers create this schema and may extend this schema. Providers cannot create conflicting entries in this schema. Assets can belong to multiple URLs at the same time, which allows us to look at it from different perspectives. URLs enable fast lookup, but do not restrict in terms of the search. This support looking at e.g. linux instances on all kinds of environments and runtimes. Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🟢 opinionated linter... Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * ✨ add method to build asset url chain from path Signed-off-by: Dominik Richter <dominik.richter@gmail.com> --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2024-03-07

⭐️ delayed asset discovery (#3496) * delayed asset discovery Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix delayed discovery scans Signed-off-by: Ivan Milchev <ivan@mondoo.com> * skip discovery only for new clients Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix build Signed-off-by: Ivan Milchev <ivan@mondoo.com> * SkipDiscovery -> DelayDiscovery Signed-off-by: Ivan Milchev <ivan@mondoo.com> * cleanup delayed discovery handling Signed-off-by: Ivan Milchev <ivan@mondoo.com> * extend provider tests to cover delayed discovery Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix comments Signed-off-by: Ivan Milchev <ivan@mondoo.com> --------- Signed-off-by: Ivan Milchev <ivan@mondoo.com>

✨ add isLastBatch to StoreResults (#3508) * ✨ add isLastBatch to StoreResults Signed-off-by: Ivan Milchev <ivan@mondoo.com> * set flag when storeresults is called Signed-off-by: Ivan Milchev <ivan@mondoo.com> --------- Signed-off-by: Ivan Milchev <ivan@mondoo.com>

2024-03-06

🧪 experimental: decayed + banded scoring (#3466) These will be added in cnspec, keywords need to be defined in cnquery Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2024-02-19

✨ refactor providers coordinator (#3308) * ✨ shutdown any unused providers on runtime.Close Signed-off-by: Ivan Milchev <ivan@mondoo.com> * add a Coordinator interface to allow mocking Signed-off-by: Ivan Milchev <ivan@mondoo.com> * use uber mockgen instead of golang mockgen Signed-off-by: Ivan Milchev <ivan@mondoo.com> * cleanup coordinator code Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix makefile and gitignore Signed-off-by: Ivan Milchev <ivan@mondoo.com> * remove Stop function from coordinator interface Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix gh actions Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix deadlock Signed-off-by: Ivan Milchev <ivan@mondoo.com> * add mocks for plugins Signed-off-by: Ivan Milchev <ivan@mondoo.com> * add more tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * refactor mock generation Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * add more tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * remove unused var Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix go.mod formatting Signed-off-by: Ivan Milchev <ivan@mondoo.com> * lock for the whole call of GetRunningProvider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * 🧹 use 1 global resource schema instead of per runtime (#3326) * use uber mockgen instead of golang mockgen Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix go.mod formatting Signed-off-by: Ivan Milchev <ivan@mondoo.com> * 🧹 use 1 global resource schema instead of per runtime Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix more tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * make sure resource overriding and extension works Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix more failing tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * code cleanup Signed-off-by: Ivan Milchev <ivan@mondoo.com> * more test fixes Signed-off-by: Ivan Milchev <ivan@mondoo.com> * more fixes Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix extensible schema test Signed-off-by: Ivan Milchev <ivan@mondoo.com> * add more safe guards for merging resources Signed-off-by: Ivan Milchev <ivan@mondoo.com> * make extensible schema test deterministic Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix comments Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix linter Signed-off-by: Ivan Milchev <ivan@mondoo.com> --------- Signed-off-by: Ivan Milchev <ivan@mondoo.com> * code cleanup Signed-off-by: Ivan Milchev <ivan@mondoo.com> --------- Signed-off-by: Ivan Milchev <ivan@mondoo.com>

2024-02-15

⭐️ SBOM pkg files (#3317) * ⭐️ add evidence of package files on disk into sbom * 🧹 improve sbom evidence rendering and simplify naming

2024-02-14

⭐️ support re-using MQL cache between provider connections (#3274) * ⭐️ support re-using MQL cache between provider connections Signed-off-by: Ivan Milchev <ivan@mondoo.com> * remove go-memoize from k8s provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * make k8s discovery reuses mql cache Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate arista provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate atlassian provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate aws provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate os provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate azure provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate equinix provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * re-use plugin.Connection instead of copying functions Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate gcp provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate github provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate gitlab provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * make sure cache is reused for discovered gcp assets Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate google-workspace provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate ipmi provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate ms365 provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate network provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * define plugin.Connection and reuse in all providers Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate okta provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate opcua provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate slack provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate terraform provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate vcd provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * migrate vsphere provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix test build Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix linter error Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix mock provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix dns tests for network provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix more tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * re-use plugin.Connection in os provider Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> * add tests for connection sharing Signed-off-by: Ivan Milchev <ivan@mondoo.com> * do not use pointer for ParentID Signed-off-by: Ivan Milchev <ivan@mondoo.com> * use plugin.Connection in winrm Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix tests Signed-off-by: Ivan Milchev <ivan@mondoo.com> --------- Signed-off-by: Ivan Milchev <ivan@mondoo.com>

2024-02-11

⭐️ refactor providers code (#3242) * refactor providers code * make sure AddRuntime uses a runtime builder function * fix k8s tests * fix runtime * migrate arista provider * migrate atlassian provider * migrate aws provider * migrate azure provider * migrate core provider * migrate equinix provider * migrate gcp provider * migrate github provider * migrate gitlab provider * migrate google-workspace provider * migrate ipmi provider * migrate ms365 provider * migrate network provider * migrate oci provider * migrate okta provider * migrate opcua provider * migrate os provider * migrate slack provider * migrate terraform provider * migrate vcd provider * migrate vsphere provider * call Disconnect on runtime close * fix broken test * fix tests * fix deadlock * properly call disconnect * add tests for service * fix terraform provider connection ids * fix more provider connection ids --------- Signed-off-by: Ivan Milchev <ivan@mondoo.com>

2024-01-30

🧹 add aws org externalid type to sbom proto (#3160)

2024-01-25

⭐ implement StoreResults for resources data (#3096)

2024-01-14

🎉 v10 🎉 (#2966) * 🛑 remove v9 from ID of providers We don't want it in the ID, since they have a separate version info. Having the ID in the name makes it much trickier to update a provider (technically). * 🧹 remove leftover _motor files from v9 migration Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🌟 v10 bump Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🟢 remove unused tracer Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🟢 lint line length in proto Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 update generated files * 🧹 fix spelling in makefile * 🧹 use list of cross-connection provider * make network provider backwards-compatible * 🧹 remove k8s.kubelet alias * 🧹 update provider ids in recordings --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com> Co-authored-by: Christoph Hartmann <chris@lollyrock.com>

2024-01-11

🧹 add started status & error message to sbom proto (#2988)

2024-01-10

⭐️ new panic handler that reports client panics (#2963) * ⭐️ new panic handler that reports client panics * 🧹 use log.Error instead of lof.Info

2023-12-28

⭐️ improved recordings

2023-12-18

add out of scope action (#2848) Signed-off-by: Ivan Milchev <ivan@mondoo.com>

2023-12-15

🐛 fix mvd go package name (#2847) Signed-off-by: Ivan Milchev <ivan@mondoo.com>

2023-12-13

🐛 Fix cvss.proto import path (#2823) Signed-off-by: Christian Zunker <christian@mondoo.com>

2023-12-11

✨ New vulnmgmt resources (#2654) * ✨ New vulnmgmt resources Signed-off-by: Christian Zunker <christian@mondoo.com> Co-authored-by: Ivan Milchev <ivan@mondoo.com>

2023-11-26

⭐️ new experimental sbom subcommand (#2642) * ⭐️ sbom support * 🧹 add experimental message to sbom sub command

2023-11-23

🧹 enable protolint (#2639)

2023-10-30

🐛 Add HealthCheck endpoint (#2419) This allows cnspec to regularly update last check-in time and version upstream. Fixes https://github.com/mondoohq/cnspec/issues/894 Signed-off-by: Christian Zunker <christian@mondoo.com>

🧹 remove v7 structs (#2395)

2023-10-26

⭐ introduce provider plugin heartbeats (#2381) * ⭐ introduce provider plugin heartbeats Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🐎 sync on provider shutdown Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 avoid log.Fatal => use provider shutdown instead Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🐛 make sure provider is not closer or shutdown Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🐎 another sync check on the heartbeat Signed-off-by: Dominik Richter <dominik.richter@gmail.com> --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-10-24

apply annotations to root asset, instead of the job config. inherit during discovery. (#2350)

2023-10-21

🐛 Use configured api-proxy (#2304) * 🐛 Use configured api-proxy Fixes #2277 Signed-off-by: Christian Zunker <christian@mondoo.com> * Fix asset.eol and vulnReport Signed-off-by: Christian Zunker <christian@mondoo.com> * 🟢 reset go.mod+sum from main Signed-off-by: Dominik Richter <dominik.richter@gmail.com> * 🧹 use pre-configured http client with proxy See: https://github.com/mondoohq/cnquery/pull/2304/files\#r1367680122 Signed-off-by: Dominik Richter <dominik.richter@gmail.com> --------- Signed-off-by: Christian Zunker <christian@mondoo.com> Signed-off-by: Dominik Richter <dominik.richter@gmail.com> Co-authored-by: Dominik Richter <dominik.richter@gmail.com>

2023-10-18

⭐ add run info (#2280) Add a new info functionality to the run-command. This collects stats about aquery you are interested in. For example: it collects information about the resources and fields that the query is going to execute: ```bash > cnquery run -c "sshd.config.params[Version] == mondoo.version" --info ``` which returns: ```bash Resources and Fields used: - sshd.config - params - mondoo - version ``` We can extend it to more information in the future. Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-10-15

🐛 set asset.fqdn for network and arista provider (#2224)

2023-10-13

🐛 fix proto go package (#2207)

2023-10-05

🧹 use new major version for go mod (#2098)

✨ introduce dev-only llx run command (#2096) v8 to v9 compatibility has bitten me once too often... actually it goes beyond that with past versions. To ease these problems, this command is introduced to make the llx bundles shareable and testable across versions: ```bash > cnquery run -c "command('echo hi 🌎').stdout.trim" --llx bundle.llx → no provider specified, defaulting to local. Use --help to see all providers. → loaded configuration from /home/zero/.config/mondoo/mondoo.yml using source default > cnquery8 run --use-llx bundle.llx → no provider specified, defaulting to local. Use --help for a list of available providers. → loaded configuration from /home/zero/.config/mondoo/mondoo.yml using source default → discover related assets for 1 asset(s) → resolved assets resolved-assets=1 command.stdout.trim: "hi 🌎" ``` Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

✨ introduce dev-only llx run command (v9 edition) (#2095) v8 to v9 compatibility has bitten me once too often... actually it goes beyond that with past versions. To ease these problems, this command is introduced to make the llx bundles shareable and testable across versions: ```bash > cnquery run -c "command('echo hi 🌎').stdout.trim" --llx bundle.llx → no provider specified, defaulting to local. Use --help to see all providers. → loaded configuration from /home/zero/.config/mondoo/mondoo.yml using source default > cnquery8 run --use-llx bundle.llx → no provider specified, defaulting to local. Use --help for a list of available providers. → loaded configuration from /home/zero/.config/mondoo/mondoo.yml using source default → discover related assets for 1 asset(s) → resolved assets resolved-assets=1 command.stdout.trim: "hi 🌎" ``` Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-10-03

✨ provide annotations upon registration. read passed in annotations from the scan cmd and send those upstream. (#2039)

🐛 add compatibility layer for v8 inventory (#2053) The `kind` field was missing to be compatible with the API. Added a layer to translate v9 kind data to the v8 kind enum (which was removed because we no longer have a centralized authority on asset kinds. Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-10-02

🧹 bring back v8 ProviderType (#2020) * change connection config backend to int32 Signed-off-by: Ivan Milchev <ivan@mondoo.com> * drop usage of Backend accross the code Signed-off-by: Ivan Milchev <ivan@mondoo.com> * fix test Signed-off-by: Ivan Milchev <ivan@mondoo.com> * bring back v8 ProviderType Signed-off-by: Ivan Milchev <ivan@mondoo.com> * do not check the value of backend Signed-off-by: Ivan Milchev <ivan@mondoo.com> --------- Signed-off-by: Ivan Milchev <ivan@mondoo.com>

2023-09-26

🐛 Add incognito flag to run command (#1912) Fixes #1889 Signed-off-by: Christian Zunker <christian@mondoo.com>

2023-09-25

✨ rename fleet => inventory (#1920) This better matches the more commonly used definition. We will keep fleet working for the next major version and then slowly deprecate it. Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-09-17

⭐ add internal mock provider (#1753) This allows us to use a recording **without connecting to any actual asset**. Assuming you have made a previous recording via e.g. ```bash cnquery scan local -f examples/os.mql.yaml --record recording.json ``` You can now use that recording with the `mock` provider in a few different ways: **1. Executing scans:** ```bash cnquery scan mock -f examples/os.mql.yaml --use-recording recording.json ```  **2. Running commands:** ```bash cnquery run mock --use-recording recording.json -c sshd.config.params ```  **3. Opening a shell:** ```bash cnquery shell mock --use-recording recording.json ```  A few follow-ups and notes: 1. The current `providers/mock` will be removed. It is now replaced by this runner. A few tests still depend on it. 3. Additionally `testutils/mockprovide` needs to be migrated to this new running. It provided a similar service but lacked the auto-detection of connections from the recording. 2. This was only implemented for a few providers (`core` and `os`), more will be migrated to support this. 4. No provider can claim the mock connector or connection type. Mock is a builtin functionality that goes hand in hand with recording. --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-09-14

📝 add dev env instructions for beta v9 (#1733) Co-authored-by: Christoph Hartmann <chris@lollyrock.com>

2023-09-13

🧹 add platform kind string to asset proto for v9 handling (#1704)

🧹 handle platform kind string on assets (v8/v9) (#1703)

2023-09-11

🧹 🧹 🧹 clean up all remaining old _resources (#1689) After https://github.com/mondoohq/cnquery/pull/1687 Big cleanups Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

🐛 fix mondoo.capabilities (#1677) 1. the detection was broken and crashed and 2. it pulled content from the os provider (sacrilege! luckily only a small file but still) There were a few options to approach this, but trying to detect this dynamically or shoving it into untyped metadata was not a good approach. Instead we now have a field attached to the connection config which can transmit capabilities per connection. We still only expose this via the `mondoo.capabilities` field which should be deprecated and replaced with a cleaner approach. Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-09-08

🧹 Add shutdown to v9 plugins (#1632) This allows us to call clean-up funcs, when we close the connection. Fixes #1628 --------- Signed-off-by: Christian Zunker <christian@mondoo.com>

2023-09-07

🐛 clarify resource extensions in LR files (#1656) We saw a race condition where the load order of providers would decide which provider became the initializer and owner of a resource. This especially led to issues with the `asset` resource, which would randomly get assigned to `core` and `os`, depending on who came first. When it was assigned to `core` that was fine and intended. But whenever it got assigned to `os` that provider would get a `connect` call with a backend that it just did not support (unless you used `os` already). For example: `cnquery run host https://... -c asset` would try to connect with the `os` provider to fill in contents for the `asset` resource, just because `os` extended it. Now we have an explicit keyword that is used to extend resources: `extend` (doh). You can see an example in the `os` provider: ``` extend asset { vulnerabilityReport() dict } ``` Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-08-31

✨ sudo for local/ssh/vagrant for v9 (#1609) Fixes #1530 Signed-off-by: Christian Zunker <christian@mondoo.com>

2023-08-28

🐛 fix providers-sdk proto health go-package (#1593) it was using the old path Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-08-22

Updating the license from MPL 2.0 to Business Source License 1.1 (#1557)

2023-08-05

🌟 add network provider (#1437) Another major migration! - migrate `tls`, `socket`, and `certificates` from `core` to a new `network` provider - make `os` depend on both `network` and `core` (it doesn't really use `core` yet, so we may kick it out later...) - migrate `parse.certificates` from `core` to `os`. It doesn't belong in `network`, since it operates on the OS. Which gets us into the fun part of this: - **cross-call resources and fields between providers**. This allows us to `parse.certificates` inside of `os` and then call into `network` to create the actual `certificate` objects - create a new `certificates` resource which is initialized from PEM and creates multiple `certificate` resources - migrate `process` and `port` to OS. It's a bit debatable if port should remain here or if we split it between `network` and `os`... There is still more resources to migrate to network, which are coming up next... --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-08-02

✨ migrate upstream config (#1417) also move upstream to providers-sdk/v1/upstream After https://github.com/mondoohq/cnquery/pull/1416 --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-07-31

⭐ migrate eol, advisories, and cves resources (#1412) This not only migrates a number of pivotal resources, it also establishes schema-merging. **Resources migration** The `platform` keyword had previously been deprecated, but we hadn't yet migrated some of its critical child resources, namely eol, advisories, and cves. These retain their previous position, despite the fact that most of platform has now been removed (in favor of asset). We will tackle these remaining fields shortly. Despite the fact that we can't remove them in v9 (due to our maintenance policy) we will target them for an eventual v10 removal, if the changes can make it in time for the upcoming release. Also, these resources have now moved from core to os, which is not intuitive and not where we want them to be long-term. In fact asset.vulnerabilityReport is also found in os! All of this is explained when we look at the next section: **Schema merging** While `core` may establish some of these principles, they cannot be implemented universally this way. To clarify: Core is the foundation to all other providers. Other providers can know about core, but core cannot know about other providers. Thus, implementing any type of cve/advisory/eol code into core is not possible, since it only knows about itself and not about anything else. However, other providers can implement the affected resources and they can even implement the same resource fields. For example, os can implement the corpus to generate EOL information about an operating system. The code for that - which is entirely OS-specific(!) - now only lives in the OS resource. We will create a few foundations in core which will serve as a base for overwriting (which has not yet happened in this PR). That said: the mechanism for overrides is there today. --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-07-24

connection overhaul (#1382) After https://github.com/mondoohq/cnquery/pull/1377 🪓 overhaul provider connections This changes a few more core aspects: 1. ParseCLI never connects to the asset. As such, there is no need to track any possible connections anymore. This also means it cannot do things that would require a connected asset, like platform detection, discovery, and resolution. All gone. This behavior was adding complexity, breaking responsibilities, and was exploding in code with what comes next... 2. Connections now only take on asset. No more inventory. Same point as above, it was adding unnecessary amount of complexity that was breaking the primariy purpose and colliding with the next point... 3. Asset connections now hold a callback. The primary use in this PR is to pull recording data from the caller. This means that we don't go through the old call-stack anymore (host:getData => plugin:create => host:recognizeID?->storeData => plugin:store) and instead this is now directed through the plugin (host:getData => plugin:create->getRecording? => host:giveRecording => plugin:store). This is critically important, because any plugin provider at any point can call CreateResource, and whenever that happens we need to be able to (a) feed it from cache, (b) record it, and (c) cache it in the plugin. This removes complexity from the caller. As a consequence, the resource generator has been overhauled. It now generates 2 methods in every resource: NewResource and CreateResource. - Only NewResource goes through argument parsing - argument parsing is now disconnected from the resource object (you can't just write to the resource anymore). See providers-sdk/v1/plugin/plugin.go for a full explanation - NewResource is used when users write something in MQL and call it (e.g. user('bob'), asset{*} etc) - CreateResource is used when resources create other resources (and thus don't want to go through all the trouble of argument parsing). It is optimized for e.g. resource.list (eg users, services, etc) Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-07-17

🐛 fix inventory tests + backend field fallback (#1364) After https://github.com/mondoohq/cnquery/pull/1363 fixed a loading panic, back to 21 tests to go Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-07-13

🧹 tidy up v9 structure for initial ci/cd testing (#1351) let's see what else we need to do to turn these tests green --------- Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

🔨 migrate vault to providers-sdk Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

🏗️ restructure around new providers-sdk We are moving everything we can from motor and resources over to providers-sdk. This also includes a few more pieces that have already moved to the providers folder. This will allow the providers folder to only hold the list of providers in the future and the providers-sdk to be used by all providers. This is merely the first step (and a big one at that), so there are plenty more to come in the next 5 days. Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-07-10

🌟 --record + --recording Allow users to record any provider and use the recording as a cache in playback. Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-07-09

🌟 end to end ssh execution Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-07-08

✨ migrated platformid to os provider Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

2023-07-07

🧹🧽 migrate and clean most other structures before rebase Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

⭐ LRv2 Signed-off-by: Dominik Richter <dominik.richter@gmail.com>