Compare commits

...

35 Commits

Author SHA1 Message Date
7be2679b91 change: Commands: host can now be localhost or 127.0.0.1 to run commands locally
All checks were successful
ci/woodpecker/push/go-lint Pipeline was successful
2025-03-21 09:09:56 -05:00
3c6e3ed914 v0.10.2
All checks were successful
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
2025-03-19 22:42:49 -05:00
02bc040e2a v0.10.2
All checks were successful
ci/woodpecker/push/go-lint Pipeline was successful
2025-03-19 22:41:18 -05:00
9f1f36215a v0.10.2 2025-03-19 22:40:06 -05:00
ff75f4bbcd feat: add variable support
All checks were successful
ci/woodpecker/push/go-lint Pipeline was successful
2025-03-16 23:43:12 -05:00
5f40713e98 feat: add variable support 2025-03-16 23:42:54 -05:00
cd5f7611a9 notifications: add http service
All checks were successful
ci/woodpecker/push/go-lint Pipeline was successful
2025-03-13 23:35:00 -05:00
b542711078 notifications: add http service 2025-03-13 23:34:37 -05:00
52dbc353e5 change: update go toolchain
All checks were successful
ci/woodpecker/push/go-lint Pipeline was successful
2025-03-12 23:22:52 -05:00
6bef0c3e5b notifications: add http config
All checks were successful
ci/woodpecker/push/go-lint Pipeline was successful
2025-03-12 10:10:45 -05:00
4d705d78fb fix: golang-ci-lint version
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
2025-03-11 22:22:39 -05:00
62d47ecfa7 fix: pipeline errors
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
2025-03-11 22:08:59 -05:00
32444ff82e fix: docs and pipeline errors
Some checks failed
ci/woodpecker/tag/gitea Pipeline failed
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline failed
2025-03-11 21:55:51 -05:00
a5a7c05640 v0.10.1
Some checks failed
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline failed
ci/woodpecker/tag/publish-docs Pipeline was successful
2025-03-11 21:37:58 -05:00
bfb81e11b2 version bump
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline was successful
2025-03-11 21:36:53 -05:00
fd4c83f9c0 Vault: keys are now referenced by name, and the actual data by data
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline failed
2025-03-11 21:33:06 -05:00
fe27c6396a LinuxUserManager: correct parameters for AddUser()
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
2025-03-11 16:13:29 -05:00
c89dde186a UserCommands: change field name
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline was successful
2025-03-11 15:37:12 -05:00
18a64de0de UserCommands: change field name 2025-03-11 15:36:43 -05:00
99c622b69f UserCommands: add field CreateUserHome 2025-03-11 15:30:07 -05:00
95e85e8b45 UserCommands: add ssh public keys when running locally
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
2025-03-11 15:21:02 -05:00
1a48c7bca5 change: create temp file when modifing password over SSH 2025-03-11 14:55:02 -05:00
5d21764ef1 fix: don't test empty env files
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
2025-03-11 13:42:40 -05:00
c7302f0e17 update docs
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline was successful
2025-03-10 12:34:33 -05:00
fb1c8ec4fb v0.10.0
Some checks failed
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
ci/woodpecker/push/go-lint Pipeline failed
2025-03-08 00:25:44 -06:00
fe9462dac0 version bump
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline was successful
2025-03-08 00:24:23 -06:00
d8453d1fb0 added external directives to Notifications, change case of keys in host, and update docs 2025-03-08 00:23:08 -06:00
65c46a1e26 add password change
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
2025-03-06 23:35:45 -06:00
f859b5961f add password change 2025-03-06 23:35:29 -06:00
25ddd65f25 v0.10.0
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
2025-03-05 00:35:14 -06:00
bcba6b2086 v0.10.0
Some checks failed
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline was successful
2025-03-03 23:46:26 -06:00
753b03861f v0.10.0 2025-03-03 23:45:28 -06:00
80a45cd595 v0.9.1
All checks were successful
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
2025-03-01 01:57:51 -06:00
551c8ad441 get S3 profile using env AWS_PROFILE 2025-03-01 01:57:33 -06:00
3823b1bf44 get S3 profile using env AWS_PROFILE 2025-03-01 01:57:05 -06:00
46 changed files with 1220 additions and 468 deletions

View File

@ -0,0 +1,3 @@
kind: Changed
body: 'Commands: `host` can now be `localhost` or `127.0.0.1` to run commands locally'
time: 2025-03-21T09:08:49.871021144-05:00

16
.changes/v0.10.0.md Normal file
View File

@ -0,0 +1,16 @@
## v0.10.0 - 2025-03-08
### Added
* Hooks: improved logging when executing
* User commands: adding SSH keys using config key `userSshPubKeys`
* directives: added support for fetching values using directive `%{externalSource:key}%`
### Changed
* Commands: if dir is not specified, run in config dir
* FileDirective: use the config directory if path is not absolute
* Host: changes to case of some keys
* Notifications: added external directive to sensitive keys
### Fixed
* LocalFetcher: return fetch error
* Lists: load file key before attempting to load from current file
* fix: host not in config file, but in ssh config, properly added to hosts struct
* SSH: password authentication bugs
* User commands: change user password works

8
.changes/v0.10.1.md Normal file
View File

@ -0,0 +1,8 @@
## v0.10.1 - 2025-03-11
### Added
* UserCommands: add ssh public keys when running locally
* UserCommands: add field CreateUserHome
### Changed
* UserCommands: create temp file when modifing password over SSH
* UserCommands: change field name
* Vault: keys are now referenced by `name`, and the actual data by `data`

6
.changes/v0.10.2.md Normal file
View File

@ -0,0 +1,6 @@
## v0.10.2 - 2025-03-19
### Added
* Notifications: http service added
* Variable support. Can be referenced with `%{var:nameOfVar}%` in select string fields.
### Changed
* vault: initialize vault before validating config

3
.changes/v0.9.1.md Normal file
View File

@ -0,0 +1,3 @@
## v0.9.1 - 2025-03-01
### Changed
* Use EnvVar AWS_PROFILE to get S3 profile

View File

@ -1,9 +1,7 @@
name: goreleaser release
steps:
golang:
image: golang:1.23
commands:
- go mod tidy
- go install github.com/goreleaser/goreleaser/v2@v2.7.0
- goreleaser release -f .goreleaser/gitea.yml --release-notes=".changes/$(go run backy.go version -V).md"
environment:

View File

@ -5,7 +5,7 @@ steps:
- go build
- go test
release:
image: golangci/golangci-lint:v1.53.3
image: golangci/golangci-lint:v1.64.7
commands:
- golangci-lint run -v --timeout 5m

View File

@ -6,6 +6,43 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html),
and is generated by [Changie](https://github.com/miniscruff/changie).
## v0.10.2 - 2025-03-19
### Added
* Notifications: http service added
* Variable support. Can be referenced with `%{var:nameOfVar}%` in select string fields.
### Changed
* vault: initialize vault before validating config
## v0.10.1 - 2025-03-11
### Added
* UserCommands: add ssh public keys when running locally
* UserCommands: add field CreateUserHome
### Changed
* UserCommands: create temp file when modifing password over SSH
* UserCommands: change field name
* Vault: keys are now referenced by `name`, and the actual data by `data`
## v0.10.0 - 2025-03-08
### Added
* Hooks: improved logging when executing
* User commands: adding SSH keys using config key `userSshPubKeys`
* directives: added support for fetching values using directive `%{externalSource:key}%`
### Changed
* Commands: if dir is not specified, run in config dir
* FileDirective: use the config directory if path is not absolute
* Host: changes to case of some keys
* Notifications: added external directive to sensitive keys
### Fixed
* LocalFetcher: return fetch error
* Lists: load file key before attempting to load from current file
* fix: host not in config file, but in ssh config, properly added to hosts struct
* SSH: password authentication bugs
* User commands: change user password works
## v0.9.1 - 2025-03-01
### Changed
* Use EnvVar AWS_PROFILE to get S3 profile
## v0.9.0 - 2025-02-28
### Added
* `list` command with subcommands `cmds` and `lists`

View File

@ -18,6 +18,7 @@
"maunium",
"mautrix",
"nikoksr",
"rawbytes",
"remotefetcher",
"Strs"
]

View File

@ -7,7 +7,7 @@ import (
"github.com/spf13/cobra"
)
const versionStr = "0.9.0"
const versionStr = "0.10.2"
var (
versionCmd = &cobra.Command{

View File

@ -15,5 +15,5 @@ The `exec` subcommand can do some things that the configuration file can't do ye
The commands have to be defined in the config file. The hosts need to at least be in the ssh_config(5) file.
```sh
backy exec host [--commands=command1 -commands=command2 ... | -c command1 -c command2 ...] [--hosts=host1 --hosts=hosts2 ... | -m host1 -c host2 ...] [flags]
backy exec host [--commands=command1 -commands=command2 ... | -c command1 -c command2 ...] [--hosts=host1 --hosts=hosts2 ... | -m host1 -m host2 ...] [flags]
```

29
docs/content/cli/list.md Normal file
View File

@ -0,0 +1,29 @@
---
title: List
---
List commands, lists, or hosts defined in config file
Usage:
```
backy list [command]
```
Available Commands:
cmds List commands defined in config file.
lists List lists defined in config file.
Flags:
```
-h, --help help for list
```
Global Flags:
```
--cmdStdOut Pass to print command output to stdout
-f, --config string config file to read from
--log-file string log file to write to
--s3-endpoint string Sets the S3 endpoint used for config file fetching. Overrides S3_ENDPOINT env variable.
-v, --verbose Sets verbose level
```

View File

@ -8,46 +8,21 @@ weight: 1
### Example Config
```yaml
commands:
stop-docker-container:
cmd: docker
Args:
- compose
- -f /some/path/to/docker-compose.yaml
- down
# if host is not defined, command will be run locally
# The host has to be defined in either the config file or the SSH Config files
host: some-host
hooks
error:
- some-other-command-when-failing
success:
- success-command
final:
- final-command
backup-docker-container-script:
cmd: /path/to/local/script
# script file is input as stdin to SSH
type: scriptFile # also can be script
environment:
- FOO=BAR
- APP=$VAR
```
{{% code file="/examples/example.yml" language="yaml" %}}
Values available for this section **(case-sensitive)**:
| name | notes | type | required
| --- | --- | --- | --- |
| `cmd` | Defines the command to execute | `string` | yes |
| `Args` | Defines the arguments to the command | `[]string` | no |
| `environment` | Defines environment variables for the command | `[]string` | no |
| `type` | See documentation further down the page. Additional fields may be required. | `string` | no |
| `getOutput` | Command(s) output is in the notification(s) | `bool` | no |
| `host` | If not specified, the command will execute locally. | `string` | no |
| `scriptEnvFile` | When type is `scriptFile` or `script`, this file is prepended to the input. | `string` | no |
| `shell` | Run the command in the shell | `string` | no |
| `hooks` | Hooks are used at the end of the individual command. Must have at least `error`, `success`, or `final`. | `map[string][]string` | no |
| name | notes | type | required | External directive support |
| ----------------| ------------------------------------------------------------------------------------------------------- | --------------------- | -------- |----------------------------|
| `cmd` | Defines the command to execute | `string` | yes | No |
| `Args` | Defines the arguments to the command | `[]string` | no | No |
| `environment` | Defines environment variables for the command | `[]string` | no | Partial |
| `type` | See documentation further down the page. Additional fields may be required. | `string` | no | No |
| `getOutput` | Command(s) output is in the notification(s) | `bool` | no | No |
| `host` | If not specified, the command will execute locally. | `string` | no | No |
| `scriptEnvFile` | When type is `scriptFile` or `script`, this file is prepended to the input. | `string` | no | No |
| `shell` | Run the command in the shell | `string` | no | No |
| `hooks` | Hooks are used at the end of the individual command. Must have at least `error`, `success`, or `final`. | `map[string][]string` | no | No |
#### cmd
@ -120,8 +95,9 @@ The following options are available:
The environment variables support expansion:
- using escaped values `$VAR` or `${VAR}`
- using any external directive, and if using the env directive, the variable will be read from a `.env` file
For now, the variables have to be defined in an `.env` file in the same directory that the program is run from.
<!-- For now, the variables expanded have to be defined in an `.env` file in the same directory that the program is run from. -->
If using it with host specified, the SSH server has to be configured to accept those env variables.

View File

@ -6,14 +6,18 @@ description: This is dedicated to user commands.
This is dedicated to `user` commands. The command `type` field must be `user`. User is a type that allows one to perform user operations. There are several additional options available when `type` is `user`:
| name | notes | type | required |
| --- | --- | --- | --- |
| `userName` | The name of a user to be configured. | `string` | yes |
| `userOperation` | The type of operation to perform. | `string` | yes |
| `userID` | The user ID to use. | `string` | yes |
| `userGroups` | The groups the user should be added to. | `[]string` | yes |
| `userShell` | The shell for the user. | `string` | yes |
| `userHome` | The user's home directory. | `string` | no |
| name | notes | type | required | External directive support
| ----------------| -------------------------------------------------------------| ---------- | ---------| --------------------------|
| `userName` | The name of a user to be configured. | `string` | yes | no |
| `userOperation` | The type of operation to perform. | `string` | yes | no |
| `userID` | The user ID to use. | `string` | no | no |
| `userGroups` | The groups the user should be added to. | `[]string` | no | no |
| `systemUser` | Create a system user. | `bool` | no | no |
| `userCreateHome`| Create the home directory. | `bool` | no | no |
| `userSshPubKeys`| The keys to add to the user's authorized keys. | `[]string` | no | yes |
| `userShell` | The shell for the user. | `string` | no | no |
| `userHome` | The user's home directory. | `string` | no | no |
| `userPassword` | The new password value when using the `password` operation. | `string` | no | yes |
#### example

View File

@ -0,0 +1,15 @@
---
title: "External Directives"
weight: 2
description: How to set up external directives.
---
External directives are for including data that should not be in the config file. The following directives are supported:
- `%{file:path/to/file}%`
- `%{env:ENV_VAR}%`
- `%{vault:vault-key}%`
See the docs of each command if the field is supported.
If the file path does not begin with a `/`, the config file's directory will be used as the starting point.

View File

@ -5,19 +5,19 @@ description: >
This page tells you how to use hosts.
---
| Key | Description | Type | Required |
|----------------------|---------------------------------------------------------------|----------|----------|
| `OS` | Operating system of the host (used for package commands) | `string` | no |
| `config` | Path to the SSH config file | `string` | no |
| `host` | Specifies the `Host` ssh_config(5) directive | `string` | yes |
| `hostname` | Hostname of the host | `string` | no |
| `knownhostsfile` | Path to the known hosts file | `string` | no |
| `port` | Port number to connect to | `uint16` | no |
| `proxyjump` | Proxy jump hosts, comma-separated | `string` | no |
| `password` | Password for SSH authentication | `string` | no |
| `privatekeypath` | Path to the private key file | `string` | no |
| `privatekeypassword` | Password for the private key file | `string` | no |
| `user` | Username for SSH authentication | `string` | no |
| Key | Description | Type | Required | External directive support |
|----------------------|---------------------------------------------------------------|----------|----------|----------------------------|
| `OS` | Operating system of the host (used for package commands) | `string` | no | No |
| `config` | Path to the SSH config file | `string` | no | No |
| `host` | Specifies the `Host` ssh_config(5) directive | `string` | yes | No |
| `hostname` | Hostname of the host | `string` | no | No |
| `knownHostsFile` | Path to the known hosts file | `string` | no | No |
| `port` | Port number to connect to | `uint16` | no | No |
| `proxyjump` | Proxy jump hosts, comma-separated | `string` | no | No |
| `password` | Password for SSH authentication | `string` | no | No |
| `privateKeyPath` | Path to the private key file | `string` | no | No |
| `privateKeyPassword` | Password for the private key file | `string` | no | Yes |
| `user` | Username for SSH authentication | `string` | no | No |
## exec host subcommand

View File

@ -39,23 +39,23 @@ There must be a section with an id (eg. `mail.test-svr`) following one of these
### mail
| key | description | type
| --- | --- | ---
| `host` | Specifies the SMTP host to connect to | `string`
| `port` | Specifies the SMTP port | `uint16`
| `senderaddress` | Address from which to send mail | `string`
| `to` | Recipients to send emails to | `[]string`
| `username` | SMTP username | `string`
| `password` | SMTP password | `string`
| key | description | type | External directive support |
| --- | --- | --- | --- |
| `host` | Specifies the SMTP host to connect to | `string` | no
| `port` | Specifies the SMTP port | `uint16` | no
| `senderaddress` | Address from which to send mail | `string` | no
| `to` | Recipients to send emails to | `[]string` | no
| `username` | SMTP username | `string` | no
| `password` | SMTP password | `string` | yes
### matrix
| key | description | type
| --- | --- | ---
| `home-server` | Specifies the Matrix server connect to | `string`
| `room-id` | Specifies the room ID of the room to send messages to | `string`
| `access-token` | Matrix access token | `string`
| `user-id` | Matrix user ID | `string`
| key | description | type | External directive support |
| --- | --- | ---| ---- |
| `home-server` | Specifies the Matrix server connect to | `string` | no
| `room-id` | Specifies the room ID of the room to send messages to | `string` | no
| `access-token` | Matrix access token | `string` | yes
| `user-id` | Matrix user ID | `string` | no
To get your access token (assumes you are using [Element](https://element.io/)) :

View File

@ -12,6 +12,10 @@ For the main config file to be fetched remotely, pass the URL using `-f [url]`.
If using S3, you should use the s3 protocol URI: `s3://bucketName/key/path`. You will also need to set the env variable `S3_ENDPOINT` to the appropriate value. The flag `--s3-endpoint` can be used to override this value or to set this value, if not already set.
## Authentication
Currently, only the AWS authentication credentials file `~/.aws/credentials` is supported. For now, the environment variable `AWS_PROFILE` is used to lookup the profile.
## Scripts
Remote script support is currently limited to http/https endpoints.

View File

@ -6,7 +6,7 @@ description: Set up and configure vault.
[Vault](https://www.vaultproject.io/) is a tool for storing secrets and other data securely.
Vault config can be used by prefixing `vault:` in front of a password or ENV var.
A Vault key can be used by prefixing `%{vault:vault.keys.name}%` in a field that supports external directives.
This is the object in the config file:
@ -18,10 +18,12 @@ vault:
keys:
- name: mongourl
mountpath: secret
key: data
path: mongo/url
type: # KVv1 or KVv2
- name:
path:
type:
mountpath:
type: KVv2 # KVv1 or KVv2
- name: someKeyName
mountpath: secret
key: keyData
type: KVv2
path: some/path
```

View File

@ -0,0 +1,108 @@
commands:
stop-docker-container:
cmd: docker
Args:
- compose
- -f /some/path/to/docker-compose.yaml
- down
# if host is not defined, cmd will be run locally
host: some-host
hooks:
final:
- hostname
error:
- hostname
backup-docker-container-script:
cmd: /path/to/script
# The host has to be defined in the config file
host: some-host
environment:
- FOO=BAR
- APP=$VAR
shell-cmd:
cmd: rsync
shell: bash
Args:
- -av some-host:/path/to/data ~/Docker/Backups/docker-data
hostname:
cmd: hostname
update-docker:
type: package
shell: zsh # best to run package commands in a shell
packageName: docker-ce
Args:
- docker-ce-cli
packageManager: apt
packageOperation: install
update-dockerApt:
# type: package
shell: zsh
cmd: apt
Args:
- update
- "&&"
- apt install -y docker-ce
- docker-ce-cli
packageManager: apt
packageOperation: install
cmd-lists:
cmds-to-run: # this can be any name you want
# all commands have to be defined
order:
- stop-docker-container
- backup-docker-container-script
- shell-cmd
- hostname
notifications:
- matrix.matrix
name: backup-some-server
cron: "0 0 1 * * *"
hostname:
name: hostname
order:
- hostname
notifications:
- mail.prod-email
hosts:
# any ssh_config(5) keys/values not listed here will be looked up in the config file or the default config file
some-host:
hostname: some-hostname
config: ~/.ssh/config
user: user
privateKeyPath: /path/to/private/key
port: 22
# can also be env:VAR
password: file:/path/to/file
# only one is supported for now
proxyjump: some-proxy-host
# optional
logging:
verbose: true
file: ./backy.log
console: false
cmd-std-out: false
notifications:
mail:
prod-email:
id: prod-email
type: mail
host: yourhost.tld
port: 587
senderAddress: email@domain.tld
to:
- admin@domain.tld
username: smtp-username@domain.tld
password: your-password-here
matrix:
matrix:
id: matrix
type: matrix
home-server: your-home-server.tld
room-id: room-id
access-token: your-access-token
user-id: your-user-id

View File

@ -0,0 +1,24 @@
commands:
stop-docker-container:
cmd: docker
Args:
- compose
- -f /some/path/to/docker-compose.yaml
- down
# if host is not defined, command will be run locally
# The host has to be defined in either the config file or the SSH Config files
host: some-host
hooks:
error:
- some-other-command-when-failing
success:
- success-command
final:
- final-command
backup-docker-container-script:
cmd: /path/to/local/script
# script file is input as stdin to SSH
type: scriptFile # also can be script
environment:
- FOO=BAR
- APP=$VAR

View File

@ -0,0 +1,3 @@
{{ $file := .Get "file" | readFile }}
{{ $lang := .Get "language" }}
{{ (print "```" $lang "\n" $file "\n```") }}

24
examples/example.yml Normal file
View File

@ -0,0 +1,24 @@
commands:
stop-docker-container:
cmd: docker
Args:
- compose
- -f /some/path/to/docker-compose.yaml
- down
# if host is not defined, command will be run locally
# The host has to be defined in either the config file or the SSH Config files
host: some-host
hooks:
error:
- some-other-command-when-failing
success:
- success-command
final:
- final-command
backup-docker-container-script:
cmd: /path/to/local/script
# script file is input as stdin to SSH
type: scriptFile # also can be script
environment:
- FOO=BAR
- APP=$VAR

8
go.mod
View File

@ -2,14 +2,13 @@ module git.andrewnw.xyz/CyberShell/backy
go 1.23
toolchain go1.23.6
replace git.andrewnw.xyz/CyberShell/backy => /home/andrew/Projects/backy
toolchain go1.23.7
require (
github.com/aws/aws-sdk-go-v2/service/s3 v1.76.0
github.com/dmarkham/enumer v1.5.11
github.com/go-co-op/gocron v1.37.0
github.com/google/uuid v1.6.0
github.com/hashicorp/vault/api v1.15.0
github.com/joho/godotenv v1.5.1
github.com/kevinburke/ssh_config v1.2.0
@ -21,6 +20,7 @@ require (
github.com/mitchellh/go-homedir v1.1.0
github.com/nikoksr/notify v1.3.0
github.com/pkg/errors v0.9.1
github.com/pkg/sftp v1.13.7
github.com/rs/zerolog v1.33.0
github.com/sethvargo/go-password v0.3.1
github.com/spf13/cobra v1.8.1
@ -50,7 +50,6 @@ require (
github.com/go-jose/go-jose/v4 v4.0.1 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
@ -65,6 +64,7 @@ require (
github.com/klauspost/compress v1.17.11 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/knadh/koanf/maps v0.1.1 // indirect
github.com/kr/fs v0.1.0 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/minio/md5-simd v1.1.2 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect

42
go.sum
View File

@ -100,6 +100,8 @@ github.com/knadh/koanf/providers/rawbytes v0.1.0 h1:dpzgu2KO6uf6oCb4aP05KDmKmAmI
github.com/knadh/koanf/providers/rawbytes v0.1.0/go.mod h1:mMTB1/IcJ/yE++A2iEZbY1MLygX7vttU+C+S/YmPu9c=
github.com/knadh/koanf/v2 v2.1.2 h1:I2rtLRqXRy1p01m/utEtpZSSA6dcJbgGVuE27kW2PzQ=
github.com/knadh/koanf/v2 v2.1.2/go.mod h1:Gphfaen0q1Fc1HTgJgSTC4oRX9R2R5ErYMZJy8fLJBo=
github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
@ -135,6 +137,8 @@ github.com/pascaldekloe/name v1.0.0/go.mod h1:Z//MfYJnH4jVpQ9wkclwu2I2MkHmXTlT9w
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/sftp v1.13.7 h1:uv+I3nNJvlKZIQGSr8JVQLNHFU9YhhNpvC14Y6KgmSM=
github.com/pkg/sftp v1.13.7/go.mod h1:KMKI0t3T6hfA+lTR/ssZdunHo+uwq7ghoN09/FSu3DY=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
@ -179,34 +183,72 @@ github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.mau.fi/util v0.8.4 h1:mVKlJcXWfVo8ZW3f4vqtjGpqtZqJvX4ETekxawt2vnQ=
go.mau.fi/util v0.8.4/go.mod h1:MOfGTs1CBuK6ERTcSL4lb5YU7/ujz09eOPVEDckuazY=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
golang.org/x/crypto v0.33.0 h1:IOBPskki6Lysi0lo9qQvbxiQ+FvsCC/YWOecCHAixus=
golang.org/x/crypto v0.33.0/go.mod h1:bVdXmD7IV/4GdElGPozy6U7lWdRXA4qyRVGJV57uQ5M=
golang.org/x/exp v0.0.0-20250207012021-f9890c6ad9f3 h1:qNgPs5exUA+G0C96DrPwNrvLSj7GT/9D+3WMWUcUg34=
golang.org/x/exp v0.0.0-20250207012021-f9890c6ad9f3/go.mod h1:tujkw807nyEEAamNbDrEGzRav+ilXA7PCRAd6xsmwiU=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.23.0 h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM=
golang.org/x/mod v0.23.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.35.0 h1:T5GQRQb2y08kTAByq9L4/bz8cipCdA8FbRTXewonqY8=
golang.org/x/net v0.35.0/go.mod h1:EglIi67kWsHKlRzzVMUD93VMSWGFOMSZgxFjparz1Qk=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w=
golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0=
golang.org/x/term v0.29.0 h1:L6pJp37ocefwRRtYPKSWOWzOtWSxVajvz2ldH/xi3iU=
golang.org/x/term v0.29.0/go.mod h1:6bl4lRlvVuDgSf3179VpIxBF0o10JUpXWOnI7nErv7s=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=
golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY=
golang.org/x/time v0.10.0 h1:3usCWA8tQn0L8+hFJQNgzpWbd89begxN66o1Ojdn5L4=
golang.org/x/time v0.10.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.30.0 h1:BgcpHewrV5AUp2G9MebG4XPFI1E2W41zU1SaqVA9vJY=
golang.org/x/tools v0.30.0/go.mod h1:c347cR/OJfw5TI+GfX7RUPNMdDRRbjvYTS0jPyvsVtY=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=

View File

@ -0,0 +1,145 @@
// Code generated by "enumer -linecomment -yaml -text -json -type=AllowedExternalDirectives"; DO NOT EDIT.
package backy
import (
"encoding/json"
"fmt"
"strings"
)
const _AllowedExternalDirectivesName = "DefaultExternalDirvaultvault-filevault-file-envfile-envfileenv"
var _AllowedExternalDirectivesIndex = [...]uint8{0, 18, 23, 33, 47, 55, 59, 62}
const _AllowedExternalDirectivesLowerName = "defaultexternaldirvaultvault-filevault-file-envfile-envfileenv"
func (i AllowedExternalDirectives) String() string {
if i < 0 || i >= AllowedExternalDirectives(len(_AllowedExternalDirectivesIndex)-1) {
return fmt.Sprintf("AllowedExternalDirectives(%d)", i)
}
return _AllowedExternalDirectivesName[_AllowedExternalDirectivesIndex[i]:_AllowedExternalDirectivesIndex[i+1]]
}
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
func _AllowedExternalDirectivesNoOp() {
var x [1]struct{}
_ = x[DefaultExternalDir-(0)]
_ = x[AllowedExternalDirectiveVault-(1)]
_ = x[AllowedExternalDirectiveVaultFile-(2)]
_ = x[AllowedExternalDirectiveAll-(3)]
_ = x[AllowedExternalDirectiveFileEnv-(4)]
_ = x[AllowedExternalDirectiveFile-(5)]
_ = x[AllowedExternalDirectiveEnv-(6)]
}
var _AllowedExternalDirectivesValues = []AllowedExternalDirectives{DefaultExternalDir, AllowedExternalDirectiveVault, AllowedExternalDirectiveVaultFile, AllowedExternalDirectiveAll, AllowedExternalDirectiveFileEnv, AllowedExternalDirectiveFile, AllowedExternalDirectiveEnv}
var _AllowedExternalDirectivesNameToValueMap = map[string]AllowedExternalDirectives{
_AllowedExternalDirectivesName[0:18]: DefaultExternalDir,
_AllowedExternalDirectivesLowerName[0:18]: DefaultExternalDir,
_AllowedExternalDirectivesName[18:23]: AllowedExternalDirectiveVault,
_AllowedExternalDirectivesLowerName[18:23]: AllowedExternalDirectiveVault,
_AllowedExternalDirectivesName[23:33]: AllowedExternalDirectiveVaultFile,
_AllowedExternalDirectivesLowerName[23:33]: AllowedExternalDirectiveVaultFile,
_AllowedExternalDirectivesName[33:47]: AllowedExternalDirectiveAll,
_AllowedExternalDirectivesLowerName[33:47]: AllowedExternalDirectiveAll,
_AllowedExternalDirectivesName[47:55]: AllowedExternalDirectiveFileEnv,
_AllowedExternalDirectivesLowerName[47:55]: AllowedExternalDirectiveFileEnv,
_AllowedExternalDirectivesName[55:59]: AllowedExternalDirectiveFile,
_AllowedExternalDirectivesLowerName[55:59]: AllowedExternalDirectiveFile,
_AllowedExternalDirectivesName[59:62]: AllowedExternalDirectiveEnv,
_AllowedExternalDirectivesLowerName[59:62]: AllowedExternalDirectiveEnv,
}
var _AllowedExternalDirectivesNames = []string{
_AllowedExternalDirectivesName[0:18],
_AllowedExternalDirectivesName[18:23],
_AllowedExternalDirectivesName[23:33],
_AllowedExternalDirectivesName[33:47],
_AllowedExternalDirectivesName[47:55],
_AllowedExternalDirectivesName[55:59],
_AllowedExternalDirectivesName[59:62],
}
// AllowedExternalDirectivesString retrieves an enum value from the enum constants string name.
// Throws an error if the param is not part of the enum.
func AllowedExternalDirectivesString(s string) (AllowedExternalDirectives, error) {
if val, ok := _AllowedExternalDirectivesNameToValueMap[s]; ok {
return val, nil
}
if val, ok := _AllowedExternalDirectivesNameToValueMap[strings.ToLower(s)]; ok {
return val, nil
}
return 0, fmt.Errorf("%s does not belong to AllowedExternalDirectives values", s)
}
// AllowedExternalDirectivesValues returns all values of the enum
func AllowedExternalDirectivesValues() []AllowedExternalDirectives {
return _AllowedExternalDirectivesValues
}
// AllowedExternalDirectivesStrings returns a slice of all String values of the enum
func AllowedExternalDirectivesStrings() []string {
strs := make([]string, len(_AllowedExternalDirectivesNames))
copy(strs, _AllowedExternalDirectivesNames)
return strs
}
// IsAAllowedExternalDirectives returns "true" if the value is listed in the enum definition. "false" otherwise
func (i AllowedExternalDirectives) IsAAllowedExternalDirectives() bool {
for _, v := range _AllowedExternalDirectivesValues {
if i == v {
return true
}
}
return false
}
// MarshalJSON implements the json.Marshaler interface for AllowedExternalDirectives
func (i AllowedExternalDirectives) MarshalJSON() ([]byte, error) {
return json.Marshal(i.String())
}
// UnmarshalJSON implements the json.Unmarshaler interface for AllowedExternalDirectives
func (i *AllowedExternalDirectives) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return fmt.Errorf("AllowedExternalDirectives should be a string, got %s", data)
}
var err error
*i, err = AllowedExternalDirectivesString(s)
return err
}
// MarshalText implements the encoding.TextMarshaler interface for AllowedExternalDirectives
func (i AllowedExternalDirectives) MarshalText() ([]byte, error) {
return []byte(i.String()), nil
}
// UnmarshalText implements the encoding.TextUnmarshaler interface for AllowedExternalDirectives
func (i *AllowedExternalDirectives) UnmarshalText(text []byte) error {
var err error
*i, err = AllowedExternalDirectivesString(string(text))
return err
}
// MarshalYAML implements a YAML Marshaler for AllowedExternalDirectives
func (i AllowedExternalDirectives) MarshalYAML() (interface{}, error) {
return i.String(), nil
}
// UnmarshalYAML implements a YAML Unmarshaler for AllowedExternalDirectives
func (i *AllowedExternalDirectives) UnmarshalYAML(unmarshal func(interface{}) error) error {
var s string
if err := unmarshal(&s); err != nil {
return err
}
var err error
*i, err = AllowedExternalDirectivesString(s)
return err
}

View File

@ -11,6 +11,7 @@ import (
"io"
"os"
"os/exec"
"strings"
"text/template"
"embed"
@ -60,7 +61,8 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
}
}
if command.Host != nil {
if !IsHostLocal(command.Host) {
outputArr, errSSH = command.RunCmdSSH(cmdCtxLogger, opts)
if errSSH != nil {
return outputArr, errSSH
@ -95,7 +97,7 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
command.Shell = "sh"
}
localCMD = exec.Command(command.Shell, command.Args...)
injectEnvIntoLocalCMD(envVars, localCMD, cmdCtxLogger)
injectEnvIntoLocalCMD(envVars, localCMD, cmdCtxLogger, opts)
cmdOutWriters = io.MultiWriter(&cmdOutBuf)
@ -144,6 +146,7 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
}
var err error
if command.Shell != "" {
cmdCtxLogger.Info().Str("Command", fmt.Sprintf("Running command %s on local machine in %s", command.Name, command.Shell)).Send()
@ -165,11 +168,17 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
}
}
if command.Type == UserCT {
if command.UserOperation == "password" {
localCMD.Stdin = command.stdin
cmdCtxLogger.Info().Str("password", command.UserPassword).Msg("user password to be updated")
}
}
if command.Dir != nil {
localCMD.Dir = *command.Dir
}
injectEnvIntoLocalCMD(envVars, localCMD, cmdCtxLogger)
injectEnvIntoLocalCMD(envVars, localCMD, cmdCtxLogger, opts)
cmdOutWriters = io.MultiWriter(&cmdOutBuf)
@ -182,24 +191,68 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
err = localCMD.Run()
outScanner := bufio.NewScanner(&cmdOutBuf)
for outScanner.Scan() {
outMap := make(map[string]interface{})
outMap["cmd"] = command.Cmd
outMap["output"] = outScanner.Text()
if str, ok := outMap["output"].(string); ok {
outputArr = append(outputArr, str)
}
// if command.GetOutput {
cmdCtxLogger.Info().Fields(outMap).Send()
// }
}
outputArr = logCommandOutput(command, cmdOutBuf, cmdCtxLogger, outputArr)
if err != nil {
cmdCtxLogger.Error().Err(fmt.Errorf("error when running cmd %s: %w", command.Name, err)).Send()
return outputArr, err
}
if command.Type == UserCT {
if command.UserOperation == "add" {
if command.UserSshPubKeys != nil {
var (
f *os.File
err error
userHome []byte
)
cmdCtxLogger.Info().Msg("adding SSH Keys")
localCMD := exec.Command(fmt.Sprintf("grep \"%s\" /etc/passwd | cut -d: -f6", command.Username))
userHome, err = localCMD.CombinedOutput()
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error finding user home from /etc/passwd: %v", err)
}
command.UserHome = strings.TrimSpace(string(userHome))
userSshDir := fmt.Sprintf("%s/.ssh", command.UserHome)
if _, err := os.Stat(userSshDir); os.IsNotExist(err) {
err := os.MkdirAll(userSshDir, 0700)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error creating directory %s %v", userSshDir, err)
}
}
if _, err := os.Stat(fmt.Sprintf("%s/authorized_keys", userSshDir)); os.IsNotExist(err) {
_, err := os.Create(fmt.Sprintf("%s/authorized_keys", userSshDir))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error creating file %s/authorized_keys: %v", userSshDir, err)
}
}
f, err = os.OpenFile(fmt.Sprintf("%s/authorized_keys", userSshDir), 0700, os.ModeAppend)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
}
defer f.Close()
for _, k := range command.UserSshPubKeys {
buf := bytes.NewBufferString(k)
cmdCtxLogger.Info().Str("key", k).Msg("adding SSH key")
if _, err := f.ReadFrom(buf); err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error adding to authorized keys: %v", err)
}
}
localCMD = exec.Command(fmt.Sprintf("chown -R %s:%s %s", command.Username, command.Username, userHome))
_, err = localCMD.CombinedOutput()
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), err
}
}
}
}
}
return outputArr, nil
}
@ -240,7 +293,7 @@ func cmdListWorker(msgTemps *msgTemplates, jobs <-chan *CmdList, results chan<-
}
// Collect output if required
if list.GetOutput || cmdToRun.GetOutput {
if list.GetOutput || cmdToRun.GetOutputInList {
outStructArr = append(outStructArr, outStruct{
CmdName: currentCmd,
CmdExecuted: currentCmd,
@ -249,17 +302,14 @@ func cmdListWorker(msgTemps *msgTemplates, jobs <-chan *CmdList, results chan<-
}
}
// Notify success if no errors occurred
if !hasError && list.NotifyConfig != nil && (list.NotifyOnSuccess || list.GetOutput) {
notifySuccess(cmdLogger, msgTemps, list, cmdsRan, outStructArr)
}
// Execute success and final hooks for all commands
for _, cmd := range list.Order {
cmdToRun := opts.Cmds[cmd]
// Execute success hooks if the command succeeded
if !hasError || cmdsRanContains(cmd, cmdsRan) {
if !hasError {
cmdToRun.ExecuteHooks("success", opts)
}
@ -276,17 +326,6 @@ func cmdListWorker(msgTemps *msgTemplates, jobs <-chan *CmdList, results chan<-
}
}
// Helper to check if a command is in the list of executed commands
func cmdsRanContains(cmd string, cmdsRan []string) bool {
for _, c := range cmdsRan {
if c == cmd {
return true
}
}
return false
}
// Helper to notify errors
func notifyError(logger zerolog.Logger, templates *msgTemplates, list *CmdList, cmdsRan []string, outStructArr []outStruct, err error, cmd *Command) {
errStruct := map[string]interface{}{
"listName": list.Name,
@ -355,7 +394,6 @@ func (opts *ConfigOpts) RunListConfig(cron string) {
result := <-results
opts.Logger.Debug().Msgf("Processing result for list %s, command %s", result.ListName, result.CmdName)
// Process final hooks for the list (already handled in worker)
}
opts.closeHostConnections()
}
@ -367,10 +405,8 @@ func (opts *ConfigOpts) ExecuteCmds() {
_, runErr := cmdToRun.RunCmd(cmdLogger, opts)
if runErr != nil {
opts.Logger.Err(runErr).Send()
cmdToRun.ExecuteHooks("error", opts)
} else {
cmdToRun.ExecuteHooks("success", opts)
}
@ -378,7 +414,6 @@ func (opts *ConfigOpts) ExecuteCmds() {
}
opts.closeHostConnections()
}
func (c *ConfigOpts) closeHostConnections() {
@ -425,27 +460,30 @@ func (cmd *Command) ExecuteHooks(hookType string, opts *ConfigOpts) {
case "error":
for _, v := range cmd.Hooks.Error {
errCmd := opts.Cmds[v]
opts.Logger.Info().Msgf("Running error hook command %s", v)
cmdLogger := opts.Logger.With().
Str("backy-cmd", v).
Str("backy-cmd", v).Str("hookType", "error").
Logger()
errCmd.RunCmd(cmdLogger, opts)
_, _ = errCmd.RunCmd(cmdLogger, opts)
}
case "success":
for _, v := range cmd.Hooks.Success {
successCmd := opts.Cmds[v]
opts.Logger.Info().Msgf("Running success hook command %s", v)
cmdLogger := opts.Logger.With().
Str("backy-cmd", v).
Str("backy-cmd", v).Str("hookType", "success").
Logger()
successCmd.RunCmd(cmdLogger, opts)
_, _ = successCmd.RunCmd(cmdLogger, opts)
}
case "final":
for _, v := range cmd.Hooks.Final {
finalCmd := opts.Cmds[v]
opts.Logger.Info().Msgf("Running final hook command %s", v)
cmdLogger := opts.Logger.With().
Str("backy-cmd", v).
Str("backy-cmd", v).Str("hookType", "final").
Logger()
finalCmd.RunCmd(cmdLogger, opts)
_, _ = finalCmd.RunCmd(cmdLogger, opts)
}
}
}
@ -455,11 +493,10 @@ func (cmd *Command) GenerateLogger(opts *ConfigOpts) zerolog.Logger {
Str("Backy-cmd", cmd.Name).Str("Host", "local machine").
Logger()
if cmd.Host != nil {
if !IsHostLocal(cmd.Host) {
cmdLogger = opts.Logger.With().
Str("Backy-cmd", cmd.Name).Str("Host", *cmd.Host).
Str("Backy-cmd", cmd.Name).Str("Host", cmd.Host).
Logger()
}
return cmdLogger
}
@ -471,7 +508,7 @@ func (opts *ConfigOpts) ExecCmdsSSH(cmdList []string, hostsList []string) {
for _, c := range cmdList {
cmd := opts.Cmds[c]
cmd.RemoteHost = host
cmd.Host = &host.Host
cmd.Host = host.Host
opts.Logger.Info().Str("host", h).Str("cmd", c).Send()
_, err := cmd.RunCmdSSH(cmd.GenerateLogger(opts), opts)
if err != nil {
@ -481,6 +518,32 @@ func (opts *ConfigOpts) ExecCmdsSSH(cmdList []string, hostsList []string) {
}
}
func logCommandOutput(command *Command, cmdOutBuf bytes.Buffer, cmdCtxLogger zerolog.Logger, outputArr []string) []string {
outScanner := bufio.NewScanner(&cmdOutBuf)
for outScanner.Scan() {
outMap := make(map[string]interface{})
outMap["cmd"] = command.Name
outMap["output"] = outScanner.Text()
if str, ok := outMap["output"].(string); ok {
outputArr = append(outputArr, str)
}
if command.OutputToLog {
cmdCtxLogger.Info().Fields(outMap).Send()
}
}
return outputArr
}
func (c *Command) GetVariablesFromConf(opts *ConfigOpts) {
c.ScriptEnvFile = replaceVarInString(opts.Vars, c.ScriptEnvFile, opts.Logger)
c.Name = replaceVarInString(opts.Vars, c.Name, opts.Logger)
c.OutputFile = replaceVarInString(opts.Vars, c.OutputFile, opts.Logger)
c.Host = replaceVarInString(opts.Vars, c.Host, opts.Logger)
}
// func executeUserCommands() []string {
// }

View File

@ -1,7 +1,6 @@
package backy
import (
"context"
"errors"
"fmt"
"net/url"
@ -22,10 +21,13 @@ import (
"github.com/rs/zerolog"
)
const macroStart string = "%{"
const macroEnd string = "}%"
const envMacroStart string = "%{env:"
const vaultMacroStart string = "%{vault:"
const (
externDirectiveStart string = "%{"
externDirectiveEnd string = "}%"
externFileDirectiveStart string = "%{file:"
envExternDirectiveStart string = "%{env:"
vaultExternDirectiveStart string = "%{vault:"
)
func (opts *ConfigOpts) InitConfig() {
var err error
@ -78,15 +80,16 @@ func (opts *ConfigOpts) InitConfig() {
logging.ExitWithMSG(fmt.Sprintf("error initializing cache: %v", err), 1, nil)
}
fetcher, err := remotefetcher.NewRemoteFetcher(opts.ConfigFilePath, opts.Cache)
if isRemoteURL(opts.ConfigFilePath) {
p, _ := getRemoteDir(opts.ConfigFilePath)
opts.ConfigDir = p
}
fetcher, err := remotefetcher.NewRemoteFetcher(opts.ConfigFilePath, opts.Cache)
if err != nil {
logging.ExitWithMSG(fmt.Sprintf("error initializing config fetcher: %v", err), 1, nil)
}
if opts.ConfigFilePath != "" {
loadConfigFile(fetcher, opts.ConfigFilePath, backyKoanf, opts)
} else {
@ -95,10 +98,83 @@ func (opts *ConfigOpts) InitConfig() {
opts.koanf = backyKoanf
}
func (opts *ConfigOpts) ReadConfig() *ConfigOpts {
setTerminalEnv()
backyKoanf := opts.koanf
if backyKoanf.Exists("variables") {
unmarshalConfigIntoStruct(backyKoanf, "variables", &opts.Vars, opts.Logger)
}
getConfigDir(opts)
opts.loadEnv()
if backyKoanf.Bool(getNestedConfig("logging", "cmd-std-out")) {
os.Setenv("BACKY_CMDSTDOUT", "enabled")
}
// override the default value of cmd-std-out if flag is set
if opts.CmdStdOut {
os.Setenv("BACKY_CMDSTDOUT", "enabled")
}
CheckConfigValues(backyKoanf, opts.ConfigFilePath)
validateExecCommandsFromCLI(backyKoanf, opts)
setLoggingOptions(backyKoanf, opts)
log := setupLogger(opts)
opts.Logger = log
log.Info().Str("config file", opts.ConfigFilePath).Send()
if err := opts.initVault(); err != nil {
log.Err(err).Send()
}
unmarshalConfigIntoStruct(backyKoanf, "commands", &opts.Cmds, opts.Logger)
getCommandEnvironments(opts)
unmarshalConfigIntoStruct(backyKoanf, "hosts", &opts.Hosts, opts.Logger)
resolveHostConfigs(opts)
for k, v := range opts.Vars {
v = getExternalConfigDirectiveValue(v, opts)
opts.Vars[k] = v
}
loadCommandLists(opts, backyKoanf)
validateCommandLists(opts)
if opts.cronEnabled && len(opts.CmdConfigLists) == 0 {
logging.ExitWithMSG("No cron fields detected in any command lists", 1, nil)
}
if err := processCmds(opts); err != nil {
logging.ExitWithMSG(err.Error(), 1, &opts.Logger)
}
filterExecuteLists(opts)
if backyKoanf.Exists("notifications") {
unmarshalConfigIntoStruct(backyKoanf, "notifications", &opts.NotificationConf, opts.Logger)
}
opts.SetupNotify()
return opts
}
func loadConfigFile(fetcher remotefetcher.RemoteFetcher, filePath string, k *koanf.Koanf, opts *ConfigOpts) {
data, err := fetcher.Fetch(filePath)
if err != nil {
logging.ExitWithMSG(fmt.Sprintf("Could not fetch config file %s: %v", filePath, err), 1, nil)
logging.ExitWithMSG(generateFileFetchErrorString(filePath, "config", err), 1, nil)
}
if err := k.Load(rawbytes.Provider(data), yaml.Parser()); err != nil {
@ -130,68 +206,6 @@ func loadDefaultConfigFiles(fetcher remotefetcher.RemoteFetcher, configFiles []s
}
}
func (opts *ConfigOpts) ReadConfig() *ConfigOpts {
setTerminalEnv()
backyKoanf := opts.koanf
opts.loadEnv()
if backyKoanf.Bool(getNestedConfig("logging", "cmd-std-out")) {
os.Setenv("BACKY_CMDSTDOUT", "enabled")
}
// override the default value of cmd-std-out if flag is set
if opts.CmdStdOut {
os.Setenv("BACKY_CMDSTDOUT", "enabled")
}
CheckConfigValues(backyKoanf, opts.ConfigFilePath)
validateCommands(backyKoanf, opts)
setLoggingOptions(backyKoanf, opts)
log := setupLogger(opts)
opts.Logger = log
log.Info().Str("config file", opts.ConfigFilePath).Send()
unmarshalConfig(backyKoanf, "commands", &opts.Cmds, opts.Logger)
validateCommandEnvironments(opts)
unmarshalConfig(backyKoanf, "hosts", &opts.Hosts, opts.Logger)
resolveHostConfigs(opts)
loadCommandLists(opts, backyKoanf)
validateCommandLists(opts)
if opts.cronEnabled && len(opts.CmdConfigLists) == 0 {
logging.ExitWithMSG("No cron fields detected in any command lists", 1, nil)
}
if err := processCmds(opts); err != nil {
logging.ExitWithMSG(err.Error(), 1, &opts.Logger)
}
filterExecuteLists(opts)
if backyKoanf.Exists("notifications") {
unmarshalConfig(backyKoanf, "notifications", &opts.NotificationConf, opts.Logger)
}
opts.SetupNotify()
if err := opts.setupVault(); err != nil {
log.Err(err).Send()
}
return opts
}
func setTerminalEnv() {
if isatty.IsTerminal(os.Stdout.Fd()) || isatty.IsCygwinTerminal(os.Stdout.Fd()) {
os.Setenv("BACKY_TERM", "enabled")
@ -200,7 +214,7 @@ func setTerminalEnv() {
}
}
func validateCommands(k *koanf.Koanf, opts *ConfigOpts) {
func validateExecCommandsFromCLI(k *koanf.Koanf, opts *ConfigOpts) {
for _, c := range opts.executeCmds {
if !k.Exists(getCmdFromConfig(c)) {
logging.ExitWithMSG(fmt.Sprintf("command %s is not in config file %s", c, opts.ConfigFilePath), 1, nil)
@ -217,6 +231,7 @@ func setLoggingOptions(k *koanf.Koanf, opts *ConfigOpts) {
logFile = k.String(getLoggingKeyFromConfig("file"))
opts.LogFilePath = logFile
}
opts.LogFilePath = logFile
zerolog.SetGlobalLevel(zerolog.InfoLevel)
if isLoggingVerbose {
@ -236,17 +251,20 @@ func setupLogger(opts *ConfigOpts) zerolog.Logger {
return zerolog.New(writers).With().Timestamp().Logger()
}
func unmarshalConfig(k *koanf.Koanf, key string, target interface{}, log zerolog.Logger) {
func unmarshalConfigIntoStruct(k *koanf.Koanf, key string, target interface{}, log zerolog.Logger) {
if err := k.UnmarshalWithConf(key, target, koanf.UnmarshalConf{Tag: "yaml"}); err != nil {
logging.ExitWithMSG(fmt.Sprintf("error unmarshalling key %s into struct: %v", key, err), 1, &log)
logging.ExitWithMSG(fmt.Sprintf("error unmarshaling key %s into struct: %v", key, err), 1, &log)
}
}
func validateCommandEnvironments(opts *ConfigOpts) {
func getCommandEnvironments(opts *ConfigOpts) {
for cmdName, cmdConf := range opts.Cmds {
if cmdConf.Env == "" {
continue
}
opts.Logger.Debug().Str("env file", cmdConf.Env).Str("cmd", cmdName).Send()
if err := testFile(cmdConf.Env); err != nil {
opts.Logger.Info().Str("cmd", cmdName).Err(err).Send()
os.Exit(1)
logging.ExitWithMSG("Could not open file"+cmdConf.Env+": "+err.Error(), 1, &opts.Logger)
}
expandEnvVars(opts.backyEnv, cmdConf.Environment)
}
@ -275,16 +293,22 @@ func resolveProxyHosts(host *Host, opts *ConfigOpts) {
}
}
func getConfigDir(opts *ConfigOpts) {
if isRemoteURL(opts.ConfigFilePath) {
p, _ := getRemoteDir(opts.ConfigFilePath)
opts.ConfigDir = p
} else {
opts.ConfigDir = path.Dir(opts.ConfigFilePath)
}
}
func loadCommandLists(opts *ConfigOpts, backyKoanf *koanf.Koanf) {
var listConfigFiles []string
var u *url.URL
var p string
// if config file is remote, use the directory of the remote file
if isRemoteURL(opts.ConfigFilePath) {
p, u = getRemoteDir(opts.ConfigFilePath)
opts.ConfigDir = p
println(p)
// // Still use local list files if a remote config file is used, but use them last
listConfigFiles = []string{u.JoinPath("lists.yml").String(), u.JoinPath("lists.yaml").String()}
} else {
opts.ConfigDir = path.Dir(opts.ConfigFilePath)
@ -304,9 +328,10 @@ func loadCommandLists(opts *ConfigOpts, backyKoanf *koanf.Koanf) {
}
if backyKoanf.Exists("cmdLists") {
unmarshalConfig(backyKoanf, "cmdLists", &opts.CmdConfigLists, opts.Logger)
if backyKoanf.Exists("cmdLists.file") {
loadCmdListsFile(backyKoanf, listsConfig, opts)
} else {
unmarshalConfigIntoStruct(backyKoanf, "cmdLists", &opts.CmdConfigLists, opts.Logger)
}
}
}
@ -346,7 +371,7 @@ func loadListConfigFile(filePath string, k *koanf.Koanf, opts *ConfigOpts) bool
return false
}
unmarshalConfig(k, "cmdLists", &opts.CmdConfigLists, opts.Logger)
unmarshalConfigIntoStruct(k, "cmdLists", &opts.CmdConfigLists, opts.Logger)
keyNotSupported("cmd-lists", "cmdLists", k, opts, true)
opts.CmdListFile = filePath
return true
@ -366,7 +391,7 @@ func loadCmdListsFile(backyKoanf *koanf.Koanf, listsConfig *koanf.Koanf, opts *C
data, err := fetcher.Fetch(opts.CmdListFile)
if err != nil {
logging.ExitWithMSG(fmt.Sprintf("Could not fetch config file %s: %v", opts.CmdListFile, err), 1, nil)
logging.ExitWithMSG(generateFileFetchErrorString(opts.CmdListFile, "list config", err), 1, nil)
}
if err := listsConfig.Load(rawbytes.Provider(data), yaml.Parser()); err != nil {
@ -374,10 +399,14 @@ func loadCmdListsFile(backyKoanf *koanf.Koanf, listsConfig *koanf.Koanf, opts *C
}
keyNotSupported("cmd-lists", "cmdLists", listsConfig, opts, true)
unmarshalConfig(listsConfig, "cmdLists", &opts.CmdConfigLists, opts.Logger)
unmarshalConfigIntoStruct(listsConfig, "cmdLists", &opts.CmdConfigLists, opts.Logger)
opts.Logger.Info().Str("using lists config file", opts.CmdListFile).Send()
}
func generateFileFetchErrorString(file, fileType string, err error) string {
return fmt.Sprintf("Could not fetch %s file %s: %v", file, fileType, err)
}
func validateCommandLists(opts *ConfigOpts) {
var cmdNotFoundSliceErr []error
for cmdListName, cmdList := range opts.CmdConfigLists {
@ -427,7 +456,7 @@ func getLoggingKeyFromConfig(key string) string {
// return fmt.Sprintf("cmdLists.%s", list)
// }
func (opts *ConfigOpts) setupVault() error {
func (opts *ConfigOpts) initVault() error {
if !opts.koanf.Bool("vault.enabled") {
return nil
}
@ -448,91 +477,34 @@ func (opts *ConfigOpts) setupVault() error {
token = os.Getenv("VAULT_TOKEN")
}
if strings.TrimSpace(token) == "" {
return fmt.Errorf("no token found, but one was required. \n\nSet the config key vault.token or the environment variable VAULT_TOKEN")
return fmt.Errorf("no token found. One is required. \n\nSet the config key vault.token or the environment variable VAULT_TOKEN")
}
client.SetToken(token)
unmarshalErr := opts.koanf.UnmarshalWithConf("vault.keys", &opts.VaultKeys, koanf.UnmarshalConf{Tag: "yaml"})
if unmarshalErr != nil {
logging.ExitWithMSG(fmt.Sprintf("error unmarshalling vault.keys into struct: %v", unmarshalErr), 1, &opts.Logger)
logging.ExitWithMSG(fmt.Sprintf("error unmarshaling vault.keys into struct: %v", unmarshalErr), 1, &opts.Logger)
}
opts.vaultClient = client
for _, v := range opts.VaultKeys {
v.Name = replaceVarInString(opts.Vars, v.Key, opts.Logger)
v.MountPath = replaceVarInString(opts.Vars, v.MountPath, opts.Logger)
}
return nil
}
func getVaultSecret(vaultClient *vault.Client, key *VaultKey) (string, error) {
var (
secret *vault.KVSecret
err error
)
if key.ValueType == "KVv2" {
secret, err = vaultClient.KVv2(key.MountPath).Get(context.Background(), key.Path)
} else if key.ValueType == "KVv1" {
secret, err = vaultClient.KVv1(key.MountPath).Get(context.Background(), key.Path)
} else if key.ValueType != "" {
return "", fmt.Errorf("type %s for key %s not known. Valid types are KVv1 or KVv2", key.ValueType, key.Name)
} else {
return "", fmt.Errorf("type for key %s must be specified. Valid types are KVv1 or KVv2", key.Name)
}
if err != nil {
return "", fmt.Errorf("unable to read secret: %v", err)
}
value, ok := secret.Data[key.Name].(string)
if !ok {
return "", fmt.Errorf("value type assertion failed: %T %#v", secret.Data[key.Name], secret.Data[key.Name])
}
return value, nil
}
func isVaultKey(str string) (string, bool) {
str = strings.TrimSpace(str)
return strings.TrimPrefix(str, "vault:"), strings.HasPrefix(str, "vault:")
}
func parseVaultKey(str string, keys []*VaultKey) (*VaultKey, error) {
keyName, isKey := isVaultKey(str)
if !isKey {
return nil, nil
}
for _, k := range keys {
if k.Name == keyName {
return k, nil
}
}
return nil, fmt.Errorf("key %s not found in vault keys", keyName)
}
func GetVaultKey(str string, opts *ConfigOpts, log zerolog.Logger) string {
key, err := parseVaultKey(str, opts.VaultKeys)
if key == nil && err == nil {
return str
}
if err != nil && key == nil {
log.Err(err).Send()
return ""
}
value, secretErr := getVaultSecret(opts.vaultClient, key)
if secretErr != nil {
log.Err(secretErr).Send()
return value
}
return value
}
func processCmds(opts *ConfigOpts) error {
// process commands
for cmdName, cmd := range opts.Cmds {
for i, v := range cmd.Args {
v = replaceVarInString(opts.Vars, v, opts.Logger)
cmd.Args[i] = v
}
if cmd.Name == "" {
cmd.Name = cmdName
}
@ -555,9 +527,13 @@ func processCmds(opts *ConfigOpts) error {
}
}
// resolve hosts
if cmd.Host != nil {
host, hostFound := opts.Hosts[*cmd.Host]
if !IsHostLocal(cmd.Host) {
cmdHost := replaceVarInString(opts.Vars, cmd.Host, opts.Logger)
if cmdHost != cmd.Host {
cmd.Host = cmdHost
}
host, hostFound := opts.Hosts[cmd.Host]
if hostFound {
cmd.RemoteHost = host
cmd.RemoteHost.Host = host.Host
@ -565,8 +541,12 @@ func processCmds(opts *ConfigOpts) error {
cmd.RemoteHost.HostName = host.HostName
}
} else {
opts.Hosts[*cmd.Host] = &Host{Host: *cmd.Host}
cmd.RemoteHost = &Host{Host: *cmd.Host}
opts.Logger.Info().Msgf("adding host %s to host list", cmd.Host)
if opts.Hosts == nil {
opts.Hosts = make(map[string]*Host)
}
opts.Hosts[cmd.Host] = &Host{Host: cmd.Host}
cmd.RemoteHost = &Host{Host: cmd.Host}
}
} else {
@ -577,10 +557,11 @@ func processCmds(opts *ConfigOpts) error {
return err
}
cmd.Dir = &cmdDir
} else {
cmd.Dir = &opts.ConfigDir
}
}
// Parse package commands
if cmd.Type == PackageCT {
if cmd.PackageManager == "" {
return fmt.Errorf("package manager is required for package command %s", cmd.PackageName)
@ -611,20 +592,34 @@ func processCmds(opts *ConfigOpts) error {
if cmd.Username == "" {
return fmt.Errorf("username is required for user command %s", cmd.Name)
}
detectOSType(cmd, opts)
var err error
cmd.Username = replaceVarInString(opts.Vars, cmd.Username, opts.Logger)
err := detectOSType(cmd, opts)
if err != nil {
opts.Logger.Info().Err(err).Str("command", cmdName).Send()
}
// Validate the operation
switch cmd.UserOperation {
case "add", "remove", "modify", "checkIfExists", "delete", "password":
cmd.userMan, err = usermanager.NewUserManager(cmd.OS)
if cmd.Host != nil {
host, ok := opts.Hosts[*cmd.Host]
if cmd.UserOperation == "password" {
opts.Logger.Debug().Msg("changing password for user: " + cmd.Username)
cmd.UserPassword = getExternalConfigDirectiveValue(cmd.UserPassword, opts)
}
if !IsHostLocal(cmd.Host) {
host, ok := opts.Hosts[cmd.Host]
if ok {
cmd.userMan, err = usermanager.NewUserManager(host.OS)
}
}
for indx, key := range cmd.UserSshPubKeys {
opts.Logger.Debug().Msg("adding SSH Keys")
key = getExternalConfigDirectiveValue(key, opts)
cmd.UserSshPubKeys[indx] = key
}
if err != nil {
return err
}
@ -656,14 +651,8 @@ func processCmds(opts *ConfigOpts) error {
return nil
}
// processHooks evaluates if hooks are valid Commands
//
// The cmd.hookRefs[hookType] is created with any hooks found.
//
// Returns an error, if any, if the hook command is not found
func processHooks(cmd *Command, hooks []string, opts *ConfigOpts, hookType string) error {
// initialize hook type
var hookCmdFound bool
cmd.hookRefs = map[string]map[string]*Command{}
cmd.hookRefs[hookType] = map[string]*Command{}
@ -690,13 +679,18 @@ func processHooks(cmd *Command, hooks []string, opts *ConfigOpts, hookType strin
}
func detectOSType(cmd *Command, opts *ConfigOpts) error {
if cmd.Host == nil {
if runtime.GOOS == "linux" { // also can be specified to FreeBSD
if IsHostLocal(cmd.Host) {
if runtime.GOOS == "linux" {
cmd.OS = "linux"
opts.Logger.Info().Msg("Unix/Linux type OS detected")
return nil
}
return fmt.Errorf("using an os that is not yet supported for user commands")
}
host, ok := opts.Hosts[*cmd.Host]
host, ok := opts.Hosts[cmd.Host]
if ok {
if host.OS != "" {
return nil
@ -728,3 +722,24 @@ func keyNotSupported(oldKey, newKey string, koanf *koanf.Koanf, opts *ConfigOpts
}
}
}
func replaceVarInString(vars map[string]string, str string, logger zerolog.Logger) string {
if strings.Contains(str, "%{var:") && strings.Contains(str, "}%") {
logger.Debug().Msgf("replacing vars in string %s", str)
for k, v := range vars {
if strings.Contains(str, "%{var:"+k+"}%") {
str = strings.ReplaceAll(str, "%{var:"+k+"}%", v)
}
}
if strings.Contains(str, "%{var:") && strings.Contains(str, "}%") {
logger.Warn().Msg("could not replace all vars in string")
}
}
return str
}
func VariadicFunctionParameterTest(allowedKeys ...string) {
if contains(allowedKeys, "file") {
println("file param included")
}
}

View File

@ -46,9 +46,9 @@ func (opts *ConfigOpts) ListCommand(cmd string) {
}
// is it remote or local
if cmdInfo.Host != nil {
if !IsHostLocal(cmdInfo.Host) {
println()
print("Host: ", *cmdInfo.Host)
print("Host: ", cmdInfo.Host)
println()
} else {

View File

@ -9,6 +9,7 @@ import (
"git.andrewnw.xyz/CyberShell/backy/pkg/logging"
"github.com/nikoksr/notify"
"github.com/nikoksr/notify/service/http"
"github.com/nikoksr/notify/service/mail"
"github.com/nikoksr/notify/service/matrix"
"maunium.net/go/mautrix/id"
@ -30,6 +31,12 @@ type MailConfig struct {
Password string `yaml:"password"`
}
type HttpConfig struct {
URL string `yaml:"url"`
Method string `yaml:"method"`
Headers map[string][]string `yaml:"headers"`
}
// SetupNotify sets up notify instances for each command list.
func (opts *ConfigOpts) SetupNotify() {
@ -58,6 +65,8 @@ func (opts *ConfigOpts) SetupNotify() {
opts.Logger.Info().Err(fmt.Errorf("error: ID %s not found in mail object", confId)).Str("list", confName).Send()
continue
}
conf.Password = getExternalConfigDirectiveValue(conf.Password, opts)
opts.Logger.Debug().Str("list", confName).Str("id", confId).Msg("adding mail notification service")
mailConf := setupMail(conf)
services = append(services, mailConf)
case "matrix":
@ -66,14 +75,23 @@ func (opts *ConfigOpts) SetupNotify() {
opts.Logger.Info().Err(fmt.Errorf("error: ID %s not found in matrix object", confId)).Str("list", confName).Send()
continue
}
conf.AccessToken = getExternalConfigDirectiveValue(conf.AccessToken, opts)
opts.Logger.Debug().Str("list", confName).Str("id", confId).Msg("adding matrix notification service")
mtrxConf, mtrxErr := setupMatrix(conf)
if mtrxErr != nil {
opts.Logger.Info().Str("list", confName).Err(fmt.Errorf("error: configuring matrix id %s failed during setup: %w", id, mtrxErr))
continue
}
// append the services
services = append(services, mtrxConf)
// service is not recognized
case "http":
conf, ok := opts.NotificationConf.HttpConfig[confId]
if !ok {
opts.Logger.Info().Err(fmt.Errorf("error: ID %s not found in http object", confId)).Str("list", confName).Send()
continue
}
opts.Logger.Debug().Str("list", confName).Str("id", confId).Msg("adding http notification service")
httpConf := setupHttp(conf)
services = append(services, httpConf)
default:
opts.Logger.Info().Err(fmt.Errorf("id %s not found", id)).Str("list", confName).Send()
}
@ -99,3 +117,19 @@ func setupMail(config MailConfig) *mail.Mail {
mailClient.BodyFormat(mail.PlainText)
return mailClient
}
func setupHttp(httpConf HttpConfig) *http.Service {
httpService := http.New()
httpService.AddReceivers(&http.Webhook{
URL: httpConf.URL,
Header: httpConf.Headers,
ContentType: "text/plain",
Method: httpConf.Method,
BuildPayload: func(subject, message string) (payload any) {
return subject + "\n\n" + message
},
})
return httpService
}

View File

@ -15,14 +15,16 @@ import (
"strings"
"time"
"github.com/google/uuid"
"github.com/kevinburke/ssh_config"
"github.com/pkg/errors"
"github.com/pkg/sftp"
"github.com/rs/zerolog"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/knownhosts"
)
var PrivateKeyExtraInfoErr = errors.New("Private key may be encrypted. \nIf encrypted, make sure the password is specified correctly in the correct section. This may be done in one of three ways: \n privatekeypassword: env:PR_KEY_PASS \n privatekeypassword: file:/path/to/password-file \n privatekeypassword: password (not recommended). \n ")
var PrivateKeyExtraInfoErr = errors.New("Private key may be encrypted. \nIf encrypted, make sure the password is specified correctly in the correct section. This may be done in one of two ways: \n Using external directives - see docs \n privatekeypassword: password (not recommended). \n ")
var TS = strings.TrimSpace
// ConnectToHost connects to a host by looking up the config values in the file ~/.ssh/config
@ -119,7 +121,6 @@ func (remoteConfig *Host) ConnectToHost(opts *ConfigOpts) error {
return errors.Wrap(err, "could not create hostkeycallback function")
}
remoteConfig.ClientConfig.HostKeyCallback = hostKeyCallback
// opts.Logger.Info().Str("user", remoteConfig.ClientConfig.User).Send()
remoteConfig.SshClient, connectErr = remoteConfig.ConnectThroughBastion(opts.Logger)
if connectErr != nil {
@ -180,11 +181,7 @@ func (remoteHost *Host) GetAuthMethods(opts *ConfigOpts) error {
return err
}
remoteHost.PrivateKeyPassword, err = GetPrivateKeyPassword(remoteHost.PrivateKeyPassword, opts, opts.Logger)
if err != nil {
return err
}
remoteHost.PrivateKeyPassword = GetPrivateKeyPassword(remoteHost.PrivateKeyPassword, opts)
if remoteHost.PrivateKeyPassword == "" {
@ -207,14 +204,13 @@ func (remoteHost *Host) GetAuthMethods(opts *ConfigOpts) error {
}
}
if remoteHost.Password == "" {
if remoteHost.Password != "" {
remoteHost.Password, err = GetPassword(remoteHost.Password, opts, opts.Logger)
opts.Logger.Debug().Str("password", remoteHost.Password).Str("Host", remoteHost.Host).Send()
if err != nil {
remoteHost.Password = GetPassword(remoteHost.Password, opts)
return err
}
// opts.Logger.Debug().Str("actual password", remoteHost.Password).Str("Host", remoteHost.Host).Send()
remoteHost.ClientConfig.Auth = append(remoteHost.ClientConfig.Auth, ssh.Password(remoteHost.Password))
}
@ -249,14 +245,13 @@ func (remoteHost *Host) GetPrivateKeyFileFromConfig() {
// If it is the port is searched in the SSH config file(s)
func (remoteHost *Host) GetPort() {
port := fmt.Sprintf("%d", remoteHost.Port)
// port specifed?
// port specified?
// port will be 0 if missing from backy config
if port == "0" {
port, _ = remoteHost.SSHConfigFile.SshConfigFile.Get(remoteHost.Host, "Port")
if port == "" {
// get port from default SSH config file
port = remoteHost.SSHConfigFile.DefaultUserSettings.Get(remoteHost.Host, "Port")
// set port to be default
@ -271,7 +266,6 @@ func (remoteHost *Host) GetPort() {
func (remoteHost *Host) CombineHostNameWithPort() {
// if the port is already in the HostName, leave it
if strings.HasSuffix(remoteHost.HostName, fmt.Sprintf(":%d", remoteHost.Port)) {
return
}
@ -321,74 +315,23 @@ func (remoteHost *Host) ConnectThroughBastion(log zerolog.Logger) (*ssh.Client,
// GetKnownHosts resolves the host's KnownHosts file if it is defined
// if not defined, the default location for this file is used
func (remotehHost *Host) GetKnownHosts() error {
func (remoteHost *Host) GetKnownHosts() error {
var knownHostsFileErr error
if TS(remotehHost.KnownHostsFile) != "" {
remotehHost.KnownHostsFile, knownHostsFileErr = getFullPathWithHomeDir(remotehHost.KnownHostsFile)
if TS(remoteHost.KnownHostsFile) != "" {
remoteHost.KnownHostsFile, knownHostsFileErr = getFullPathWithHomeDir(remoteHost.KnownHostsFile)
return knownHostsFileErr
}
remotehHost.KnownHostsFile, knownHostsFileErr = getFullPathWithHomeDir("~/.ssh/known_hosts")
remoteHost.KnownHostsFile, knownHostsFileErr = getFullPathWithHomeDir("~/.ssh/known_hosts")
return knownHostsFileErr
}
func GetPrivateKeyPassword(key string, opts *ConfigOpts, log zerolog.Logger) (string, error) {
var prKeyPassword string
if strings.HasPrefix(key, "file:") {
privKeyPassFilePath := strings.TrimPrefix(key, "file:")
privKeyPassFilePath, _ = getFullPathWithHomeDir(privKeyPassFilePath)
keyFile, keyFileErr := os.Open(privKeyPassFilePath)
if keyFileErr != nil {
return "", errors.Errorf("Private key password file %s failed to open. \n Make sure it is accessible and correct.", privKeyPassFilePath)
}
passwordScanner := bufio.NewScanner(keyFile)
for passwordScanner.Scan() {
prKeyPassword = passwordScanner.Text()
}
} else if strings.HasPrefix(key, "env:") {
privKey := strings.TrimPrefix(key, "env:")
privKey = strings.TrimPrefix(privKey, "${")
privKey = strings.TrimSuffix(privKey, "}")
privKey = strings.TrimPrefix(privKey, "$")
prKeyPassword = os.Getenv(privKey)
} else {
prKeyPassword = key
}
prKeyPassword = GetVaultKey(prKeyPassword, opts, opts.Logger)
return prKeyPassword, nil
func GetPrivateKeyPassword(key string, opts *ConfigOpts) string {
return getExternalConfigDirectiveValue(key, opts)
}
// GetPassword gets any password
func GetPassword(pass string, opts *ConfigOpts, log zerolog.Logger) (string, error) {
pass = strings.TrimSpace(pass)
if pass == "" {
return "", nil
}
var password string
if strings.HasPrefix(pass, "file:") {
passFilePath := strings.TrimPrefix(pass, "file:")
passFilePath, _ = getFullPathWithHomeDir(passFilePath)
keyFile, keyFileErr := os.Open(passFilePath)
if keyFileErr != nil {
return "", errors.New("Password file failed to open")
}
passwordScanner := bufio.NewScanner(keyFile)
for passwordScanner.Scan() {
password = passwordScanner.Text()
}
} else if strings.HasPrefix(pass, "env:") {
passEnv := strings.TrimPrefix(pass, "env:")
passEnv = strings.TrimPrefix(passEnv, "${")
passEnv = strings.TrimSuffix(passEnv, "}")
passEnv = strings.TrimPrefix(passEnv, "$")
password = os.Getenv(passEnv)
} else {
password = pass
}
password = GetVaultKey(password, opts, opts.Logger)
return password, nil
func GetPassword(pass string, opts *ConfigOpts) string {
return getExternalConfigDirectiveValue(pass, opts)
}
func (remoteConfig *Host) GetProxyJumpFromConfig(hosts map[string]*Host) error {
@ -489,7 +432,6 @@ func (command *Command) RunCmdSSH(cmdCtxLogger zerolog.Logger, opts *ConfigOpts)
env: command.Environment,
}
)
// Getting the command type must be done before concatenating the arguments
command = getCommandTypeAndSetCommandInfo(command)
// Prepare command arguments
@ -499,8 +441,8 @@ func (command *Command) RunCmdSSH(cmdCtxLogger zerolog.Logger, opts *ConfigOpts)
cmdCtxLogger.Info().
Str("Command", command.Name).
Str("Host", *command.Host).
Msgf("Running %s on host %s", getCommandTypeAndSetCommandInfoLabel(command.Type), *command.Host)
Str("Host", command.Host).
Msgf("Running %s on host %s", getCommandTypeAndSetCommandInfoLabel(command.Type), command.Host)
// cmdCtxLogger.Debug().Str("cmd", command.Cmd).Strs("args", command.Args).Send()
@ -565,10 +507,97 @@ func (command *Command) RunCmdSSH(cmdCtxLogger zerolog.Logger, opts *ConfigOpts)
ArgsStr = fmt.Sprintf("%s %s", command.Cmd, ArgsStr)
}
cmdCtxLogger.Debug().Str("cmd + args", ArgsStr).Send()
// Run simple command
if command.Type == UserCT && command.UserOperation == "password" {
// cmdCtxLogger.Debug().Msgf("adding stdin")
userNamePass := fmt.Sprintf("%s:%s", command.Username, command.UserPassword)
client, err := sftp.NewClient(command.RemoteHost.SshClient)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error creating sftp client: %v", err)
}
uuidFile := uuid.New()
passFilePath := fmt.Sprintf("/tmp/%s", uuidFile.String())
passFile, passFileErr := client.Create(passFilePath)
if passFileErr != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error creating file /tmp/%s: %v", uuidFile.String(), passFileErr)
}
_, err = passFile.Write([]byte(userNamePass))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error writing to file /tmp/%s: %v", uuidFile.String(), err)
}
ArgsStr = fmt.Sprintf("cat %s | chpasswd", passFilePath)
defer passFile.Close()
rmFileFunc := func() {
_ = client.Remove(passFilePath)
}
defer rmFileFunc()
// commandSession.Stdin = command.stdin
}
if err := commandSession.Run(ArgsStr); err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error running command: %w", err)
}
if command.Type == UserCT {
if command.UserOperation == "add" {
if command.UserSshPubKeys != nil {
var (
f *sftp.File
err error
userHome []byte
client *sftp.Client
)
cmdCtxLogger.Info().Msg("adding SSH Keys")
commandSession, _ = command.RemoteHost.createSSHSession(opts)
userHome, err = commandSession.CombinedOutput(fmt.Sprintf("grep \"%s\" /etc/passwd | cut -d: -f6", command.Username))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error finding user home from /etc/passwd: %v", err)
}
command.UserHome = strings.TrimSpace(string(userHome))
userSshDir := fmt.Sprintf("%s/.ssh", command.UserHome)
client, err = sftp.NewClient(command.RemoteHost.SshClient)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error creating sftp client: %v", err)
}
err = client.MkdirAll(userSshDir)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error creating directory %s: %v", userSshDir, err)
}
_, err = client.Create(fmt.Sprintf("%s/authorized_keys", userSshDir))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
}
f, err = client.OpenFile(fmt.Sprintf("%s/authorized_keys", userSshDir), os.O_APPEND|os.O_CREATE|os.O_WRONLY)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
}
defer f.Close()
for _, k := range command.UserSshPubKeys {
buf := bytes.NewBufferString(k)
cmdCtxLogger.Info().Str("key", k).Msg("adding SSH key")
if _, err := f.ReadFrom(buf); err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), fmt.Errorf("error adding to authorized keys: %v", err)
}
}
commandSession, _ = command.RemoteHost.createSSHSession(opts)
_, err = commandSession.CombinedOutput(fmt.Sprintf("chown -R %s:%s %s", command.Username, command.Username, userHome))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), err
}
}
}
}
}
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), nil
@ -622,7 +651,7 @@ func (command *Command) runScript(session *ssh.Session, cmdCtxLogger zerolog.Log
return collectOutput(outputBuf, command.Name, cmdCtxLogger, true), fmt.Errorf("error waiting for shell: %w", err)
}
return collectOutput(outputBuf, command.Name, cmdCtxLogger, command.GetOutput), nil
return collectOutput(outputBuf, command.Name, cmdCtxLogger, command.OutputToLog), nil
}
// runScriptFile handles the execution of script files.
@ -782,3 +811,8 @@ func CheckIfHostHasHostName(host string) (bool, string) {
println(HostName)
return HostName != "", HostName
}
func IsHostLocal(host string) bool {
host = strings.ToLower(host)
return host == "127.0.0.1" || host == "localhost" || host == ""
}

View File

@ -26,15 +26,15 @@ type (
ConfigFilePath string `yaml:"config,omitempty"`
Host string `yaml:"host,omitempty"`
HostName string `yaml:"hostname,omitempty"`
KnownHostsFile string `yaml:"knownhostsfile,omitempty"`
KnownHostsFile string `yaml:"knownHostsFile,omitempty"`
ClientConfig *ssh.ClientConfig
SSHConfigFile *sshConfigFile
SshClient *ssh.Client
Port uint16 `yaml:"port,omitempty"`
ProxyJump string `yaml:"proxyjump,omitempty"`
Password string `yaml:"password,omitempty"`
PrivateKeyPath string `yaml:"privatekeypath,omitempty"`
PrivateKeyPassword string `yaml:"privatekeypassword,omitempty"`
PrivateKeyPath string `yaml:"privateKeyPath,omitempty"`
PrivateKeyPassword string `yaml:"privateKeyPassword,omitempty"`
useDefaultConfig bool
User string `yaml:"user,omitempty"`
isProxyHost bool
@ -51,44 +51,30 @@ type (
Command struct {
Name string `yaml:"name,omitempty"`
// command to run
Cmd string `yaml:"cmd"`
// See CommandType enum further down the page for acceptable values
Type CommandType `yaml:"type,omitempty"`
// host on which to run cmd
Host *string `yaml:"host,omitempty"`
Host string `yaml:"host,omitempty"`
// Hooks are for running commands on certain events
Hooks *Hooks `yaml:"hooks,omitempty"`
// hook refs are internal references of commands for each hook type
hookRefs map[string]map[string]*Command
// Shell specifies which shell to run the command in, if any.
Shell string `yaml:"shell,omitempty"`
RemoteHost *Host `yaml:"-"`
// Args is an array that holds the arguments to cmd
Args []string `yaml:"args,omitempty"`
/*
Dir specifies a directory in which to run the command.
*/
Dir *string `yaml:"dir,omitempty"`
// Env points to a file containing env variables to be used with the command
Env string `yaml:"env,omitempty"`
// Environment holds env variables to be used with the command
Environment []string `yaml:"environment,omitempty"`
// Output determines if output is requested.
//
// Only for when command is in a list.
GetOutput bool `yaml:"getOutput,omitempty"`
GetOutputInList bool `yaml:"getOutputInList,omitempty"`
ScriptEnvFile string `yaml:"scriptEnvFile"`
@ -102,10 +88,8 @@ type (
PackageName string `yaml:"packageName,omitempty"`
// Version specifies the desired version for package execution
PackageVersion string `yaml:"packageVersion,omitempty"`
// PackageOperation specifies the action for package-related commands (e.g., "install" or "remove")
PackageOperation PackageOperation `yaml:"packageOperation,omitempty"`
pkgMan pkgman.PackageManager
@ -113,42 +97,37 @@ type (
packageCmdSet bool
// END PACKAGE COMMAND FIELDS
// RemoteSource specifies a URL to fetch the command or configuration remotely
RemoteSource string `yaml:"remoteSource,omitempty"`
// FetchBeforeExecution determines if the remoteSource should be fetched before running
FetchBeforeExecution bool `yaml:"fetchBeforeExecution,omitempty"`
Fetcher remotefetcher.RemoteFetcher
// BEGIN USER COMMAND FIELDS
// Username specifies the username for user creation or related operations
Username string `yaml:"userName,omitempty"`
UserID string `yaml:"userID,omitempty"`
// UserGroups specifies the groups to add the user to
UserGroups []string `yaml:"userGroups,omitempty"`
// UserHome specifies the home directory for the user
UserHome string `yaml:"userHome,omitempty"`
// UserShell specifies the shell for the user
UserShell string `yaml:"userShell,omitempty"`
// SystemUser specifies whether the user is a system account
SystemUser bool `yaml:"systemUser,omitempty"`
UserCreateHome bool `yaml:"userCreateHome,omitempty"`
UserIsSystem bool `yaml:"userIsSystem,omitempty"`
// UserPassword specifies the password for the user (can be file: or plain text)
UserPassword string `yaml:"userPassword,omitempty"`
UserSshPubKeys []string `yaml:"userSshPubKeys,omitempty"`
userMan usermanager.UserManager
// OS for the command, only used when type is user
OS string `yaml:"OS,omitempty"`
// UserOperation specifies the action for user-related commands (e.g., "create" or "remove")
UserOperation string `yaml:"userOperation,omitempty"`
userCmdSet bool
@ -226,6 +205,8 @@ type (
List ListConfig
Vars map[string]string `yaml:"variables"`
VaultKeys []*VaultKey `yaml:"keys"`
koanf *koanf.Koanf
@ -244,6 +225,7 @@ type (
VaultKey struct {
Name string `yaml:"name"`
Key string `yaml:"key"`
Path string `yaml:"path"`
ValueType string `yaml:"type"`
MountPath string `yaml:"mountpath"`
@ -259,6 +241,7 @@ type (
Notifications struct {
MailConfig map[string]MailConfig `yaml:"mail,omitempty"`
MatrixConfig map[string]MatrixStruct `yaml:"matrix,omitempty"`
HttpConfig map[string]HttpConfig `yaml:"http,omitempty"`
}
CmdOutput struct {
@ -295,8 +278,9 @@ type (
}
// use ints so we can use enums
CommandType int
PackageOperation int
CommandType int
PackageOperation int
AllowedExternalDirectives int
)
//go:generate go run github.com/dmarkham/enumer -linecomment -yaml -text -json -type=CommandType
@ -319,3 +303,14 @@ const (
PackOpCheckVersion // checkVersion
PackOpIsInstalled // isInstalled
)
//go:generate go run github.com/dmarkham/enumer -linecomment -yaml -text -json -type=AllowedExternalDirectives
const (
DefaultExternalDir AllowedExternalDirectives = iota
AllowedExternalDirectiveVault // vault
AllowedExternalDirectiveVaultFile // vault-file
AllowedExternalDirectiveAll // vault-file-env
AllowedExternalDirectiveFileEnv // file-env
AllowedExternalDirectiveFile // file
AllowedExternalDirectiveEnv // env
)

View File

@ -6,6 +6,7 @@ package backy
import (
"bytes"
"context"
"errors"
"fmt"
"os"
@ -16,6 +17,7 @@ import (
"git.andrewnw.xyz/CyberShell/backy/pkg/logging"
"git.andrewnw.xyz/CyberShell/backy/pkg/remotefetcher"
vault "github.com/hashicorp/vault/api"
"github.com/joho/godotenv"
"github.com/knadh/koanf/v2"
"github.com/rs/zerolog"
@ -108,7 +110,11 @@ func injectEnvIntoSSH(envVarsToInject environmentVars, process *ssh.Session, opt
goto errEnvFile
}
for key, val := range envMap {
process.Setenv(key, GetVaultKey(val, opts, log))
err = process.Setenv(key, GetVaultKey(val, opts, log))
if err != nil {
log.Error().Err(err).Send()
}
}
}
@ -119,12 +125,16 @@ errEnvFile:
if strings.Contains(envVal, "=") {
envVarArr := strings.Split(envVal, "=")
process.Setenv(envVarArr[0], GetVaultKey(envVarArr[1], opts, log))
err := process.Setenv(envVarArr[0], getExternalConfigDirectiveValue(envVarArr[1], opts))
if err != nil {
log.Error().Err(err).Send()
}
}
}
}
func injectEnvIntoLocalCMD(envVarsToInject environmentVars, process *exec.Cmd, log zerolog.Logger) {
func injectEnvIntoLocalCMD(envVarsToInject environmentVars, process *exec.Cmd, log zerolog.Logger, opts *ConfigOpts) {
if envVarsToInject.file != "" {
envPath, _ := getFullPathWithHomeDir(envVarsToInject.file)
@ -148,7 +158,8 @@ errEnvFile:
for _, envVal := range envVarsToInject.env {
if strings.Contains(envVal, "=") {
process.Env = append(process.Env, envVal)
envVarArr := strings.Split(envVal, "=")
process.Env = append(process.Env, fmt.Sprintf("%s=%s", envVarArr[0], getExternalConfigDirectiveValue(envVarArr[1], opts)))
}
}
process.Env = append(process.Env, os.Environ()...)
@ -181,7 +192,6 @@ func testFile(c string) error {
return fileOpenErr
}
}
return nil
}
@ -247,11 +257,9 @@ func (opts *ConfigOpts) loadEnv() {
opts.backyEnv = backyEnv
}
// expandEnvVars expands environment variables with the env used in the config
func expandEnvVars(backyEnv map[string]string, envVars []string) {
env := func(name string) string {
name = strings.ToUpper(name)
envVar, found := backyEnv[name]
if found {
return envVar
@ -259,16 +267,15 @@ func expandEnvVars(backyEnv map[string]string, envVars []string) {
return ""
}
// parse env variables using new macros
for indx, v := range envVars {
if strings.HasPrefix(v, macroStart) && strings.HasSuffix(v, macroEnd) {
if strings.HasPrefix(v, envMacroStart) {
v = strings.TrimPrefix(v, envMacroStart)
v = strings.TrimRight(v, macroEnd)
out, _ := shell.Expand(v, env)
envVars[indx] = out
}
if strings.HasPrefix(v, envExternDirectiveStart) && strings.HasSuffix(v, externDirectiveEnd) {
v = strings.TrimPrefix(v, envExternDirectiveStart)
v = strings.TrimRight(v, externDirectiveEnd)
out, _ := shell.Expand(v, env)
envVars[indx] = out
}
}
}
@ -296,7 +303,8 @@ func getCommandTypeAndSetCommandInfo(command *Command) *Command {
command.Username,
command.UserHome,
command.UserShell,
command.SystemUser,
command.UserIsSystem,
command.UserCreateHome,
command.UserGroups,
command.Args)
case "modify":
@ -324,7 +332,7 @@ func parsePackageVersion(output string, cmdCtxLogger zerolog.Logger, command *Co
// println(output)
if err != nil {
cmdCtxLogger.Error().Err(err).Str("package", command.PackageName).Msg("Error parsing package version output")
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.GetOutput), err
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.OutputToLog), err
}
cmdCtxLogger.Info().
@ -349,3 +357,97 @@ func parsePackageVersion(output string, cmdCtxLogger zerolog.Logger, command *Co
}
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, false), err
}
func getExternalConfigDirectiveValue(key string, opts *ConfigOpts) string {
if !(strings.HasPrefix(key, externDirectiveStart) && strings.HasSuffix(key, externDirectiveEnd)) {
return key
}
key = replaceVarInString(opts.Vars, key, opts.Logger)
opts.Logger.Debug().Str("expanding external key", key).Send()
if strings.HasPrefix(key, envExternDirectiveStart) {
key = strings.TrimPrefix(key, envExternDirectiveStart)
key = strings.TrimSuffix(key, externDirectiveEnd)
key = os.Getenv(key)
}
if strings.HasPrefix(key, externFileDirectiveStart) {
var err error
var keyValue []byte
key = strings.TrimPrefix(key, externFileDirectiveStart)
key = strings.TrimSuffix(key, externDirectiveEnd)
key, err = getFullPathWithHomeDir(key)
if err != nil {
opts.Logger.Err(err).Send()
return ""
}
if !path.IsAbs(key) {
key = path.Join(opts.ConfigDir, key)
}
keyValue, err = os.ReadFile(key)
if err != nil {
opts.Logger.Err(err).Send()
return ""
}
key = string(keyValue)
}
if strings.HasPrefix(key, vaultExternDirectiveStart) {
key = strings.TrimPrefix(key, vaultExternDirectiveStart)
key = strings.TrimSuffix(key, externDirectiveEnd)
key = GetVaultKey(key, opts, opts.Logger)
}
return key
}
func getVaultSecret(vaultClient *vault.Client, key *VaultKey) (string, error) {
var (
secret *vault.KVSecret
err error
)
if key.ValueType == "KVv2" {
secret, err = vaultClient.KVv2(key.MountPath).Get(context.Background(), key.Path)
} else if key.ValueType == "KVv1" {
secret, err = vaultClient.KVv1(key.MountPath).Get(context.Background(), key.Path)
} else if key.ValueType != "" {
return "", fmt.Errorf("type %s for key %s not known. Valid types are KVv1 or KVv2", key.ValueType, key.Name)
} else {
return "", fmt.Errorf("type for key %s must be specified. Valid types are KVv1 or KVv2", key.Name)
}
if err != nil {
return "", fmt.Errorf("unable to read secret: %v", err)
}
value, ok := secret.Data[key.Key].(string)
if !ok {
return "", fmt.Errorf("value type assertion failed for vault key %s: %T %#v", key.Name, secret.Data[key.Name], secret.Data[key.Name])
}
return value, nil
}
func getVaultKeyData(keyName string, keys []*VaultKey) (*VaultKey, error) {
for _, k := range keys {
if k.Name == keyName {
return k, nil
}
}
return nil, fmt.Errorf("key %s not found in vault keys", keyName)
}
func GetVaultKey(str string, opts *ConfigOpts, log zerolog.Logger) string {
key, err := getVaultKeyData(str, opts.VaultKeys)
if key == nil && err == nil {
return str
}
if err != nil && key == nil {
log.Err(err).Send()
return ""
}
value, secretErr := getVaultSecret(opts.vaultClient, key)
if secretErr != nil {
log.Err(secretErr).Send()
return value
}
return value
}

View File

@ -116,7 +116,7 @@ func (c *Cache) Set(source, hash string, data []byte, dataType string) (CacheDat
path := filepath.Join(c.dir, fmt.Sprintf("%s-%s", fileName, sourceHash))
if _, err := os.Stat(path); os.IsNotExist(err) {
os.MkdirAll(c.dir, 0700)
_ = os.MkdirAll(c.dir, 0700)
}
err := os.WriteFile(path, data, 0644)
@ -171,7 +171,7 @@ func (cf *CachedFetcher) Hash(data []byte) string {
func LoadMetadataFromFile(filePath string) ([]*CacheData, error) {
if _, err := os.Stat(filePath); os.IsNotExist(err) {
// Create the file if it does not exist
os.MkdirAll(path.Dir(filePath), 0700)
_ = os.MkdirAll(path.Dir(filePath), 0700)
emptyData := []byte("[]")
err := os.WriteFile(filePath, emptyData, 0644)
if err != nil {

View File

@ -29,7 +29,7 @@ func NewRemoteFetcher(source string, cache *Cache, options ...FetcherOption) (Re
option(&config)
}
// If FileType is empty (i.e. WithFileType was not called), yaml is the default file type
// WithFileType was not called. yaml is the default file type
if strings.TrimSpace(config.FileType) == "" {
config.FileType = "yaml"
}

View File

@ -20,7 +20,7 @@ func (l *LocalFetcher) Fetch(source string) ([]byte, error) {
if l.config.IgnoreFileNotFound {
return nil, ErrIgnoreFileNotFound
}
return nil, nil
return nil, err
}
file, err := os.Open(source)
if err != nil {

View File

@ -44,9 +44,8 @@ func NewS3Fetcher(endpoint string, options ...FetcherOption) (*S3Fetcher, error)
*/
s3Endpoint := os.Getenv("S3_ENDPOINT")
creds, err := getS3Credentials("default", s3Endpoint, cfg.HTTPClient)
creds, err := getS3Credentials(os.Getenv("AWS_PROFILE"), s3Endpoint, cfg.HTTPClient)
if err != nil {
println(err.Error())
return nil, err
}
@ -133,7 +132,7 @@ func getS3Credentials(profile, host string, httpClient *http.Client) (*credentia
if hdirErr != nil {
return nil, hdirErr
}
s3Creds := credentials.NewFileAWSCredentials(path.Join(homeDir, ".aws", "credentials"), "default")
s3Creds := credentials.NewFileAWSCredentials(path.Join(homeDir, ".aws", "credentials"), profile)
credVals, credErr := s3Creds.GetWithContext(&credentials.CredContext{Endpoint: host, Client: httpClient})
if credErr != nil {
return nil, credErr

View File

@ -15,7 +15,7 @@ func (l LinuxUserManager) NewLinuxManager() *LinuxUserManager {
}
// AddUser adds a new user to the system.
func (l LinuxUserManager) AddUser(username, homeDir, shell string, isSystem bool, groups, args []string) (string, []string) {
func (l LinuxUserManager) AddUser(username, homeDir, shell string, isSystem, createHome bool, groups, args []string) (string, []string) {
baseArgs := []string{}
if isSystem {
@ -38,6 +38,10 @@ func (l LinuxUserManager) AddUser(username, homeDir, shell string, isSystem bool
baseArgs = append(baseArgs, args...)
}
if createHome {
baseArgs = append(baseArgs, "-m")
}
args = append(baseArgs, username)
cmd := "useradd"

View File

@ -10,7 +10,7 @@ import (
// UserManager defines the interface for user management operations.
// All functions but one return a string for the command and any args.
type UserManager interface {
AddUser(username, homeDir, shell string, isSystem bool, groups, args []string) (string, []string)
AddUser(username, homeDir, shell string, createHome, isSystem bool, groups, args []string) (string, []string)
RemoveUser(username string) (string, []string)
ModifyUser(username, homeDir, shell string, groups []string) (string, []string)
// Modify password uses chpasswd for Linux systems to build the command to change the password

View File

@ -1,7 +1,8 @@
#!/bin/bash
set -eou pipefail
go mod tidy
go generate ./...
export CURRENT_TAG="$(go run backy.go version -V)"
CURRENT_TAG="$(go run backy.go version -V)"
goreleaser -f .goreleaser/github.yml check
goreleaser -f .goreleaser/gitea.yml check
changie batch $CURRENT_TAG

1
tests/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
private

17
tests/ErrorHook.yml Normal file
View File

@ -0,0 +1,17 @@
commands:
echoTestFail:
cmd: ech
shell: bash
Args: hello world
hooks:
error:
- errorCmd
errorCmd:
name: get docker version
cmd: docker
getOutput: true
outputToLog: true
Args:
- "-v"
host: email-svr

14
tests/HookNotInFile.yaml Normal file
View File

@ -0,0 +1,14 @@
commands:
echoTestFail:
cmd: ech
shell: bash
Args: hello world
hooks:
error:
- errorCm #
errorCmd:
name: get docker version
cmd: docker
Args:
- "-v"

16
tests/SuccessHook.yml Normal file
View File

@ -0,0 +1,16 @@
commands:
echoTestSuccess:
cmd: echo
shell: bash
Args: hello world
hooks:
success:
- successCmd
errorCmd:
name: get docker version
cmd: docker
getOutput: true
outputToLog: true
Args:
- "-v"

27
tests/VaultTest.yml Normal file
View File

@ -0,0 +1,27 @@
commands:
vaultEnvVar:
cmd: echo
shell: /bin/zsh
Args:
- ${VAULT_VAR}
environment:
"VAULT_VAR=%{vault:vaultTestSecret}%"
logging:
verbose: true
vault:
token: root
address: http://127.0.0.1:8200
enabled: true
keys:
- name: vaultTestSecret
key: data
mountpath: secret
path: test/var
type: KVv2 # KVv1 or KVv2
cmdLists:
addUsers:
order:
- vaultEnvVar

View File

@ -1,18 +0,0 @@
commands:
echoTestPass:
cmd: echo
shell: bash
Args: hello world
runRemoteShellScriptSuccess:
cmd:
packageCommandSuccess:
packageName: docker-ce
Args:
- docker-ce-cli
packageManager: apt
packageOperation: install