Compare commits

..
14 Commits
Author SHA1 Message Date
CyberShell 5f6459d5be v0.13.1
ci/woodpecker/push/publish-docs Pipeline is pending
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
2026-08-15 09:53:03 -05:00
CyberShell 1d2a3a8306 v0.13.0
ci/woodpecker/push/publish-docs Pipeline failed
ci/woodpecker/tag/gitea Pipeline failed
ci/woodpecker/tag/publish-docs Pipeline failed
2026-08-15 00:35:57 -05:00
CyberShell 045dedef67 v0.13.0
ci/woodpecker/push/go-lint Pipeline failed
2026-08-15 00:34:13 -05:00
CyberShell 60989a3905 fix error hooks and add listing everything
ci/woodpecker/push/go-lint Pipeline failed
2026-08-15 00:30:55 -05:00
CyberShell e789924547 fix error hooks and add listing everything
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline failed
2026-08-15 00:16:05 -05:00
CyberShell 62ac2934dc v0.12.1
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
ci/woodpecker/push/go-lint Pipeline failed
2026-03-23 11:30:32 -05:00
CyberShell c078632691 remove shell history boolean
ci/woodpecker/push/publish-docs Pipeline was successful
2026-02-14 18:57:37 -06:00
CyberShell 52e25aad77 v0.12.0
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
2026-02-11 13:18:46 -06:00
CyberShell 9f996f60c6 internal logic handling for cron webserver 2026-02-11 13:18:06 -06:00
CyberShell 4c152f8089 internal logic handling for cron webserver 2026-02-11 13:17:15 -06:00
CyberShell 4d2e4ce533 v0.11.5
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
2026-02-10 12:01:11 -06:00
CyberShell 9e3960ce9f Command.Type: scriptFile no longer requests psudoterminal
ci/woodpecker/push/go-lint Pipeline failed
2026-02-10 12:00:08 -06:00
CyberShell b2d89352a3 v0.11.4
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
ci/woodpecker/push/go-lint Pipeline failed
ci/woodpecker/push/publish-docs Pipeline was successful
2026-02-01 07:21:20 -06:00
CyberShell 765ef2ee36 v0.11.3
ci/woodpecker/push/publish-docs Pipeline was successful
ci/woodpecker/tag/gitea Pipeline was successful
ci/woodpecker/release/publish-docs Pipeline was successful
ci/woodpecker/tag/publish-docs Pipeline was successful
2026-01-31 01:06:18 -06:00
50 changed files with 1217 additions and 256 deletions
+7
View File
@@ -0,0 +1,7 @@
## v0.11.3 - 2026-01-31
### Added
* Command: saveShellHistory for scriptFile commands over SSH
* Starting on Variables and Templates
### Changed
* File output for commands now adds hostname to beginning of filename
* Testing: docker testing infra
+3
View File
@@ -0,0 +1,3 @@
## v0.11.4 - 2026-02-01
### Changed
* Command.[name].output.file: now appends correctly to the beginning of file in an absolute path
+3
View File
@@ -0,0 +1,3 @@
## v0.11.5 - 2026-02-10
### Changed
* Command.Type: scriptFile no longer requests psudoterminal
+3
View File
@@ -0,0 +1,3 @@
## v0.12.0 - 2026-02-11
### Changed
* internal logic handling for cron webserver
+8
View File
@@ -0,0 +1,8 @@
## v0.12.1 - 2026-03-23
### Added
* Command output respects `command.[name].output.toLog` when standard output is enabled
### Changed
* SSH: revert scriptFile to before 0.11.3
* Output: tabs print as 4 spaces instead of '\t'
### Fixed
* Internal: output buffer will be copied to temporary buffer
+5
View File
@@ -0,0 +1,5 @@
## v0.13.0 - 2026-08-15
### Changed
* List subcommands print out everything with no arguments
### Fixed
* Error hooks no longer return after one is executed
+3
View File
@@ -0,0 +1,3 @@
## v0.13.1 - 2026-08-15
### Changed
* Print help for exec command if no args given
+2
View File
@@ -34,6 +34,8 @@ snapshot:
version_template: "{{ incpatch .Version }}-next"
changelog:
disable: false
release:
prerelease: auto
gitea_urls:
api: https://git.andrewnw.xyz/api/v1
+2
View File
@@ -31,6 +31,8 @@ archives:
formats: [zip]
checksum:
name_template: 'checksums.txt'
release:
prerelease: auto
snapshot:
version_template: "{{ incpatch .Version }}-next"
changelog:
+10 -1
View File
@@ -10,5 +10,14 @@
"nikoksr",
"Strs"
],
"CodeGPT.apiKey": "CodeGPT Plus Beta"
"CodeGPT.apiKey": "CodeGPT Plus Beta",
"yaml.schemas": {
"file:///c%3A/Users/anw12/.vscode/extensions/continue.continue-1.2.14-win32-x64/config-yaml-schema.json": [
".continue/**/*.yaml"
],
"file:///c%3A/Users/anw12/.vscode/extensions/continue.continue-1.2.16-win32-x64/config-yaml-schema.json": [
".continue/**/*.yaml"
],
"file:///c:/Users/anw12/.vscode/extensions/continue.continue-1.2.16-win32-x64/config-yaml-schema.json": "vscode-local:/c%3A/Users/anw12/.continue/config.yaml"
}
}
+39
View File
@@ -6,6 +6,45 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html),
and is generated by [Changie](https://github.com/miniscruff/changie).
## v0.13.1 - 2026-08-15
### Changed
* Print help for exec command if no args given
## v0.13.0 - 2026-08-15
### Changed
* List subcommands print out everything with no arguments
### Fixed
* Error hooks no longer return after one is executed
## v0.12.1 - 2026-03-23
### Added
* Command output respects `command.[name].output.toLog` when standard output is enabled
### Changed
* SSH: revert scriptFile to before 0.11.3
* Output: tabs print as 4 spaces instead of '\t'
### Fixed
* Internal: output buffer will be copied to temporary buffer
## v0.12.0 - 2026-02-11
### Changed
* internal logic handling for cron webserver
## v0.11.5 - 2026-02-10
### Changed
* Command.Type: scriptFile no longer requests psudoterminal
## v0.11.4 - 2026-02-01
### Changed
* Command.[name].output.file: now appends correctly to the beginning of file in an absolute path
## v0.11.3 - 2026-01-31
### Added
* Command: saveShellHistory for scriptFile commands over SSH
* Starting on Variables and Templates
### Changed
* File output for commands now adds hostname to beginning of filename
* Testing: docker testing infra
## v0.11.2 - 2025-12-27
### Added
* Upgraded GoCron; web ui viewer for viewing cron jobs
+3 -1
View File
@@ -5,6 +5,8 @@
package cmd
import (
"fmt"
"git.andrewnw.xyz/CyberShell/backy/pkg/backy"
"git.andrewnw.xyz/CyberShell/backy/pkg/logging"
@@ -29,7 +31,7 @@ func execute(cmd *cobra.Command, args []string) {
parseS3Config()
if len(args) < 1 {
logging.ExitWithMSG("Please provide a command to run. Pass --help to see options.", 1, nil)
logging.ExitWithMSG(fmt.Sprintf("Please provide a command to run.\n\n%s", cmd.Long), 1, nil)
}
opts := backy.NewConfigOptions(configFile,
+1 -1
View File
@@ -9,7 +9,7 @@ import (
var (
hostExecCommand = &cobra.Command{
Use: "host [--command=command1 --command=command2 ... | -c command1 -c command2 ...] [--hosts=host1 --hosts=hosts2 ... | -m host1 -m host2 ...] ",
Short: "Runs command defined in config file on the hosts in order specified.",
Short: "Specify command(s) defined in config file on the host in order specified.",
Long: "Host executes specified commands on the hosts defined in config file.\nUse the --commands or -c flag to choose the commands.",
Run: Host,
}
+42 -28
View File
@@ -5,8 +5,10 @@
package cmd
import (
"fmt"
"os"
"git.andrewnw.xyz/CyberShell/backy/pkg/backy"
"git.andrewnw.xyz/CyberShell/backy/pkg/logging"
"github.com/spf13/cobra"
)
@@ -15,28 +17,26 @@ var (
listCmd = &cobra.Command{
Use: "list [command]",
Short: "List commands, lists, or hosts defined in config file.",
Long: "List commands, lists, or hosts defined in config file",
Long: "List commands, lists, or hosts defined in config file. The subcommands take zero or more arguments to print specific commands or lists",
}
listCmds = &cobra.Command{
Use: "cmds [cmd1 cmd2 cmd3...]",
Short: "List commands defined in config file.",
Long: "List commands defined in config file",
Short: "Prints commands defined in config file.",
Long: "Prints commands defined in config file. Pass no arguments to print all commands",
Run: ListCommands,
}
listCmdLists = &cobra.Command{
Use: "lists [list1 list2 ...]",
Short: "List lists defined in config file.",
Long: "List lists defined in config file",
Short: "Prints lists defined in config file.",
Long: "Prints lists defined in config file. Pass no arguments to print all lists",
Run: ListCommandLists,
}
)
var listsToList []string
var cmdsToList []string
func init() {
listCmd.AddCommand(listCmds, listCmdLists)
parseS3Config()
}
@@ -46,13 +46,6 @@ func ListCommands(cmd *cobra.Command, args []string) {
// - cmds
// - lists
// - if none, list all commands
if len(args) > 0 {
cmdsToList = args
} else {
logging.ExitWithMSG("Error: list cmds subcommand needs commands to list", 1, nil)
}
parseS3Config()
opts := backy.NewConfigOptions(configFile,
backy.SetLogFile(logFile),
@@ -61,21 +54,28 @@ func ListCommands(cmd *cobra.Command, args []string) {
opts.InitConfig()
opts.ParseConfigurationFile()
for _, v := range cmdsToList {
opts.ListCommand(v)
if len(args) > 0 {
for _, v := range args {
opts.ListCommand(v)
}
os.Exit(0)
}
if len(opts.Cmds) == 0 {
fmt.Println("No commands defined in config file")
os.Exit(1)
}
for c := range opts.Cmds {
println()
println()
println("---------------------------------------------------------------------------------")
opts.ListCommand(c)
}
}
func ListCommandLists(cmd *cobra.Command, args []string) {
parseS3Config()
if len(args) > 0 {
listsToList = args
} else {
logging.ExitWithMSG("Error: lists subcommand needs lists", 1, nil)
}
opts := backy.NewConfigOptions(configFile,
backy.SetLogFile(logFile),
backy.SetHostsConfigFile(hostsConfigFile))
@@ -83,8 +83,22 @@ func ListCommandLists(cmd *cobra.Command, args []string) {
opts.InitConfig()
opts.ParseConfigurationFile()
for _, v := range listsToList {
opts.ListCommandList(v)
if len(args) > 0 {
for _, v := range args {
opts.ListCommandList(v)
}
os.Exit(0)
}
if len(opts.CmdConfigLists) == 0 {
fmt.Println("No command lists defined in config file")
os.Exit(1)
}
for c := range opts.CmdConfigLists {
println()
println()
println("---------------------------------------------------------------------------------")
opts.ListCommandList(c)
}
}
+20
View File
@@ -0,0 +1,20 @@
package cmd
import "testing"
// validate checks commands passed in from the command line
/*
*args:
* -t - type of cmd
* -c
*/
func TestUserCmd(t *testing.T) {
/*
*/
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"github.com/spf13/cobra"
)
const versionStr = "0.11.2"
const versionStr = "0.13.1"
var (
versionCmd = &cobra.Command{
+6 -6
View File
@@ -88,7 +88,7 @@ Usage:
backy exec [command]
Available Commands:
host Runs command defined in config file on the hosts in order specified.
host Specify command(s) defined in config file on the host in order specified.
hosts Runs command defined in config file on the hosts in order specified.
Flags:
@@ -153,14 +153,14 @@ Global Flags:
## list
```
List commands, lists, or hosts defined in config file
List commands, lists, or hosts defined in config file. The subcommands take zero or more arguments to print specific commands or lists
Usage:
backy list [command]
Available Commands:
cmds List commands defined in config file.
lists List lists defined in config file.
cmds Prints commands defined in config file.
lists Prints lists defined in config file.
Flags:
-h, --help help for list
@@ -178,7 +178,7 @@ Use "backy list [command] --help" for more information about a command.
## list cmds
```
List commands defined in config file
Prints commands defined in config file. Pass no arguments to print all commands
Usage:
backy list cmds [cmd1 cmd2 cmd3...] [flags]
@@ -197,7 +197,7 @@ Global Flags:
## list lists
```
List lists defined in config file
Prints lists defined in config file. Pass no arguments to print all lists
Usage:
backy list lists [list1 list2 ...] [flags]
+13 -4
View File
@@ -2,14 +2,23 @@
title: Exec
---
The `exec` subcommand can do some things that the configuration file can't do yet. The command `exec host` can execute commands on many hosts.
~~The `exec` subcommand can do some things that the configuration file can't do yet. The command `exec host` can execute commands on many hosts.~~
{{% notice info %}}
For now only the `exec host` sub-command is implemented. More to come.
{{% /notice %}}
{{% notice warning %}}
The config file's `hosts` are overridden by this command. Hooks are not altered. For consistent results, create a local config file with only the hosts you want to use.
{{% /notice %}}
`exec host` takes the following arguments:
```sh
-c, --commands strings Accepts space-separated names of commands.
-h, --help help for host
-m, --hosts strings Accepts space-separated names of hosts.
-c, --command stringArray Accepts space-separated names of commands. Specify multiple times for multiple commands.
-h, --help help for host
-m, --hosts stringArray Accepts space-separated names of hosts. Specify multiple times for multiple hosts.
```
The commands have to be defined in the config file. The hosts need to at least be in the ssh_config(5) file.
+3 -3
View File
@@ -3,7 +3,7 @@ title: List
---
List commands, lists, or hosts defined in config file
List commands, lists, or hosts defined in config file. The subcommands take zero or more arguments to print specific commands or lists
Usage:
```
@@ -11,8 +11,8 @@ Usage:
```
Available Commands:
cmds List commands defined in config file.
lists List lists defined in config file.
cmds Prints commands defined in config file.
lists Prints lists defined in config file.
Flags:
```
Executable
+5
View File
@@ -0,0 +1,5 @@
#!/bin/bash
go install .
systemctl --user stop backy
cp ~/go/bin/backy ~/prodConfigs/backups/backy/backy
systemctl --user start backy
+94 -56
View File
@@ -18,6 +18,7 @@ import (
"embed"
"github.com/rs/zerolog"
"gopkg.in/natefinch/lumberjack.v2"
)
//go:embed templates/*.txt
@@ -65,10 +66,10 @@ func (e *PackageCommandExecutor) Run(cmd *Command, opts *ConfigOpts, logger zero
// Execute the package version command
execCmd := exec.Command(cmd.Cmd, cmd.Args...)
cmdOutWriters = io.MultiWriter(&cmdOutBuf)
if IsCmdStdOutEnabled() {
cmdOutWriters = io.MultiWriter(os.Stdout, &cmdOutBuf)
var err error
cmdOutWriters, _, err = makeCmdOutWriters(&cmdOutBuf, "")
if err != nil {
return nil, err
}
execCmd.Stdout = cmdOutWriters
execCmd.Stderr = cmdOutWriters
@@ -145,6 +146,27 @@ func (e *LocalCommandExecutor) Run(cmd *Command, opts *ConfigOpts, logger zerolo
return outputArr, nil
}
// makeCmdOutWriters constructs an io.Writer that writes to the provided buffer,
// optionally also to stdout and/or a file. If a file path is provided the
// caller is responsible for closing the returned *os.File when non-nil.
func makeCmdOutWriters(buf *bytes.Buffer, outputFile string) (io.Writer, *os.File, error) {
writers := io.MultiWriter(buf)
if outputFile != "" {
fileLogger := &lumberjack.Logger{
MaxSize: 50, // megabytes
MaxBackups: 3,
MaxAge: 28, //days
Compress: false, // disabled by default
}
fileLogger.Filename = outputFile
writers = io.MultiWriter(fileLogger, writers)
}
return writers, nil, nil
}
// ensureRemoteHost ensures localCmd.RemoteHost is set for the given host.
// It prefers opts.Hosts lookup and falls back to a minimal Host entry so remote execution can proceed.
func (opts *ConfigOpts) ensureRemoteHost(localCmd *Command, host string) {
@@ -158,7 +180,7 @@ func (opts *ConfigOpts) ensureRemoteHost(localCmd *Command, host string) {
}
}
// fallback: create a minimal Host so RunCmdOnHost sees a non-nil RemoteHost.
// This uses host as the address/alias; further fields (user/key) will use defaults.
// This uses host as the address/alias; further fields (user/key) will use be looked up as needed.
localCmd.RemoteHost = &Host{Host: host}
}
@@ -183,11 +205,13 @@ func (opts *ConfigOpts) ExecCommandOnHostsParallel(cmdName string) ([]CmdResult,
// shallow copy to avoid races
local := *cmdObj
local.Host = h
opts.Logger.Debug().Str("host", h).Msg("executing command in parallel on host")
opts.Logger.Info().Str("host", h).Msg("executing command in parallel on host")
local.cmdLoggers.global = opts.Logger
var err error
if IsHostLocal(h) {
_, err := local.RunCmd(local.GenerateLogger(opts), opts)
local.GenerateLogger(opts)
_, err := local.RunCmd(local.cmdLoggers.cmdContxt, opts)
resultsCh <- CmdResult{CmdName: cmdName, ListName: "", Error: err}
return
// _, err = local.RunCmd(local.GenerateLogger(opts), opts)
@@ -196,9 +220,10 @@ func (opts *ConfigOpts) ExecCommandOnHostsParallel(cmdName string) ([]CmdResult,
// ensure RemoteHost is populated before calling RunCmdOnHost
opts.ensureRemoteHost(&local, h)
_, err = local.RunCmdOnHost(local.GenerateLogger(opts), opts)
local.GenerateLogger(opts)
_, err = local.RunCmdOnHost(local.cmdLoggers.cmdContxt, opts)
resultsCh <- CmdResult{CmdName: cmdName, ListName: "", Error: err}
resultsCh <- CmdResult{CmdName: cmdName, ListName: "", Error: err, HostName: h}
}(host)
}
@@ -208,6 +233,9 @@ func (opts *ConfigOpts) ExecCommandOnHostsParallel(cmdName string) ([]CmdResult,
var results []CmdResult
for r := range resultsCh {
results = append(results, r)
if r.Error != nil {
opts.Logger.Info().AnErr("error", r.Error).Str("cmd", r.CmdName).Send()
}
}
return results, nil
}
@@ -238,7 +266,11 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
cmdCtxLogger.Warn().Msg("both 'host' and 'hosts' are set; 'hosts' will be ignored")
return nil, fmt.Errorf("both 'host' and 'hosts' are set; please set one or the other")
} else if command.Hosts != nil {
opts.ExecCommandOnHostsParallel(command.Name)
_, err := opts.ExecCommandOnHostsParallel(command.Name)
if err != nil {
opts.Logger.Err(err).Send()
return nil, err
}
return nil, nil
}
@@ -250,7 +282,9 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
}
if command.Type == UserCommandType {
if command.UserOperation == "password" {
command.cmdLoggers.global.Info().Str("password", command.UserPassword).Msg("user password to be updated")
cmdCtxLogger.Info().Str("password", command.UserPassword).Msg("user password to be updated")
}
}
@@ -283,23 +317,13 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
localCMD = exec.Command(command.Shell, command.Args...)
injectEnvIntoLocalCMD(envVars, localCMD, cmdCtxLogger, opts)
cmdOutWriters = io.MultiWriter(&cmdOutBuf)
if IsCmdStdOutEnabled() {
cmdOutWriters = io.MultiWriter(os.Stdout, &cmdOutBuf)
var outFile *os.File
cmdOutWriters, outFile, err := makeCmdOutWriters(&cmdOutBuf, command.Output.File)
if err != nil {
return nil, err
}
if command.Output.File != "" {
file, err := os.Create(command.Output.File)
if err != nil {
return nil, fmt.Errorf("error creating output file: %w", err)
}
defer file.Close()
cmdOutWriters = io.MultiWriter(file, &cmdOutBuf)
if IsCmdStdOutEnabled() {
cmdOutWriters = io.MultiWriter(os.Stdout, file, &cmdOutBuf)
}
if outFile != nil {
defer outFile.Close()
}
localCMD.Stdin = bytes.NewReader(script)
@@ -370,10 +394,9 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
injectEnvIntoLocalCMD(envVars, localCMD, cmdCtxLogger, opts)
cmdOutWriters = io.MultiWriter(&cmdOutBuf)
if IsCmdStdOutEnabled() {
cmdOutWriters = io.MultiWriter(os.Stdout, &cmdOutBuf)
cmdOutWriters, _, err = makeCmdOutWriters(&cmdOutBuf, "")
if err != nil {
return outputArr, err
}
localCMD.Stdout = cmdOutWriters
@@ -402,7 +425,7 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
localCMD := exec.Command(fmt.Sprintf("grep \"%s\" /etc/passwd | cut -d: -f6", command.Username))
userHome, err = localCMD.CombinedOutput()
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error finding user home from /etc/passwd: %v", err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error finding user home from /etc/passwd: %v", err)
}
command.UserHome = strings.TrimSpace(string(userHome))
@@ -411,33 +434,33 @@ func (command *Command) RunCmd(cmdCtxLogger zerolog.Logger, opts *ConfigOpts) ([
if _, err := os.Stat(userSshDir); os.IsNotExist(err) {
err := os.MkdirAll(userSshDir, 0700)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error creating directory %s %v", userSshDir, err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error creating directory %s %v", userSshDir, err)
}
}
if _, err := os.Stat(fmt.Sprintf("%s/authorized_keys", userSshDir)); os.IsNotExist(err) {
_, err := os.Create(fmt.Sprintf("%s/authorized_keys", userSshDir))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error creating file %s/authorized_keys: %v", userSshDir, err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error creating file %s/authorized_keys: %v", userSshDir, err)
}
}
authorizedKeysFile, err = os.OpenFile(fmt.Sprintf("%s/authorized_keys", userSshDir), 0700, os.ModeAppend)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
}
defer authorizedKeysFile.Close()
for _, k := range command.UserSshPubKeys {
buf := bytes.NewBufferString(k)
cmdCtxLogger.Info().Str("key", k).Msg("adding SSH key")
if _, err := authorizedKeysFile.ReadFrom(buf); err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error adding to authorized keys: %v", err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error adding to authorized keys: %v", err)
}
}
localCMD = exec.Command(fmt.Sprintf("chown -R %s:%s %s", command.Username, command.Username, userHome))
_, err = localCMD.CombinedOutput()
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), err
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), err
}
}
@@ -461,10 +484,10 @@ func cmdListWorker(msgTemps *msgTemplates, jobs <-chan *CmdList, results chan<-
commandExecuted = cmdToRun
currentCmd := cmdToRun.Name
fieldsMap["cmd"] = currentCmd
cmdLogger = cmdToRun.GenerateLogger(opts)
cmdLogger.Info().Fields(fieldsMap).Send()
cmdToRun.GenerateLogger(opts)
cmdToRun.cmdLoggers.cmdContxt.Info().Fields(fieldsMap).Send()
outputArr, runErr := cmdToRun.RunCmd(cmdLogger, opts)
outputArr, runErr := cmdToRun.RunCmd(cmdToRun.cmdLoggers.cmdContxt, opts)
cmdsRan = append(cmdsRan, cmd)
if runErr != nil {
@@ -525,10 +548,10 @@ func cmdListWorkerWithHosts(msgTemps *msgTemplates, jobs <-chan *CmdList, hosts
commandExecuted = cmdToRun
currentCmd := cmdToRun.Name
fieldsMap["cmd"] = currentCmd
cmdLogger = cmdToRun.GenerateLogger(opts)
cmdLogger.Info().Fields(fieldsMap).Send()
cmdToRun.GenerateLogger(opts)
cmdToRun.cmdLoggers.cmdContxt.Info().Fields(fieldsMap).Send()
outputArr, runErr := cmdToRun.RunCmd(cmdLogger, opts)
outputArr, runErr := cmdToRun.RunCmd(cmdToRun.cmdLoggers.cmdContxt, opts)
cmdsRan = append(cmdsRan, cmd)
if runErr != nil {
@@ -671,8 +694,8 @@ func cmdListWorkerExecuteCommandsInParallel(msgTemps *msgTemplates, jobs <-chan
cmdToRun.Host = host.Host
cmdToRun.RemoteHost = host
}
cmdLogger = cmdToRun.GenerateLogger(opts)
cmdLogger.Info().Fields(fieldsMap).Send()
cmdToRun.GenerateLogger(opts)
cmdToRun.cmdLoggers.cmdContxt.Info().Fields(fieldsMap).Send()
print("Running cmd on: ", host.Host, "\n")
go func(cmd string, host *Host) {
@@ -680,7 +703,7 @@ func cmdListWorkerExecuteCommandsInParallel(msgTemps *msgTemplates, jobs <-chan
currentCmd := cmdToRun.Name
fieldsMap["cmd"] = currentCmd
outputArr, runErr := cmdToRun.RunCmd(cmdLogger, opts)
outputArr, runErr := cmdToRun.RunCmd(cmdToRun.cmdLoggers.cmdContxt, opts)
if runErr != nil {
cmdLogger.Err(runErr).Send()
cmdToRun.ExecuteHooks("error", opts)
@@ -856,8 +879,8 @@ func (opts *ConfigOpts) ExecuteListOnHosts(lists []string, parallel bool) {
func (opts *ConfigOpts) ExecuteCmds() {
for _, cmd := range opts.executeCmds {
cmdToRun := opts.Cmds[cmd]
cmdLogger := cmdToRun.GenerateLogger(opts)
_, runErr := cmdToRun.RunCmd(cmdLogger, opts)
cmdToRun.GenerateLogger(opts)
_, runErr := cmdToRun.RunCmd(cmdToRun.cmdLoggers.cmdContxt, opts)
if runErr != nil {
opts.Logger.Err(runErr).Send()
cmdToRun.ExecuteHooks("error", opts)
@@ -921,7 +944,6 @@ func (cmd *Command) ExecuteHooks(hookType string, opts *ConfigOpts) {
cmdLogger.Info().Msgf("Running error hook command %s", v)
// URGENT: Never returns
_, _ = errCmd.RunCmd(cmdLogger, opts)
return
}
case "success":
@@ -945,17 +967,29 @@ func (cmd *Command) ExecuteHooks(hookType string, opts *ConfigOpts) {
}
}
func (cmd *Command) GenerateLogger(opts *ConfigOpts) zerolog.Logger {
cmdLogger := opts.Logger.With().
func (cmd *Command) GenerateLogger(opts *ConfigOpts) {
cmd.cmdLoggers.cmdContxt = opts.Logger.With().
Str("Backy-cmd", cmd.Name).Str("Host", "local machine").
Logger()
if !IsHostLocal(cmd.Host) {
cmdLogger = opts.Logger.With().
cmd.cmdLoggers.cmdContxt = opts.Logger.With().
Str("Backy-cmd", cmd.Name).Str("Host", cmd.Host).
Logger()
}
return cmdLogger
}
func (cmd *Command) GenerateLoggerForCmd(logger zerolog.Logger) zerolog.Logger {
cmd.cmdLoggers.cmdContxt = logger.With().
Str("Backy-cmd", cmd.Name).Str("Host", "local machine").
Logger()
if !IsHostLocal(cmd.Host) {
cmd.cmdLoggers.cmdContxt = logger.With().
Str("Backy-cmd", cmd.Name).Str("Host", cmd.Host).
Logger()
}
return cmd.cmdLoggers.cmdContxt
}
func (opts *ConfigOpts) ExecCmdsOnHosts(cmdList []string, hostsList []string) {
@@ -967,7 +1001,8 @@ func (opts *ConfigOpts) ExecCmdsOnHosts(cmdList []string, hostsList []string) {
cmd.RemoteHost = host
cmd.Host = h
if IsHostLocal(h) {
_, err := cmd.RunCmd(cmd.GenerateLogger(opts), opts)
cmd.GenerateLogger(opts)
_, err := cmd.RunCmd(cmd.cmdLoggers.cmdContxt, opts)
if err != nil {
opts.Logger.Err(err).Str("host", h).Str("cmd", c).Send()
}
@@ -975,7 +1010,8 @@ func (opts *ConfigOpts) ExecCmdsOnHosts(cmdList []string, hostsList []string) {
cmd.Host = host.Host
opts.Logger.Info().Str("host", h).Str("cmd", c).Send()
_, err := cmd.RunCmdOnHost(cmd.GenerateLogger(opts), opts)
cmd.GenerateLogger(opts)
_, err := cmd.RunCmdOnHost(cmd.cmdLoggers.cmdContxt, opts)
if err != nil {
opts.Logger.Err(err).Str("host", h).Str("cmd", c).Send()
}
@@ -994,7 +1030,8 @@ func (opts *ConfigOpts) ExecCmdsOnHostsInParallel(cmdList []string, hostsList []
cmd.RemoteHost = host
cmd.Host = h
if IsHostLocal(h) {
_, err := cmd.RunCmd(cmd.GenerateLogger(opts), opts)
cmd.GenerateLogger(opts)
_, err := cmd.RunCmd(cmd.cmdLoggers.cmdContxt, opts)
if err != nil {
opts.Logger.Err(err).Str("host", h).Str("cmd", c).Send()
}
@@ -1002,7 +1039,8 @@ func (opts *ConfigOpts) ExecCmdsOnHostsInParallel(cmdList []string, hostsList []
cmd.Host = host.Host
opts.Logger.Info().Str("host", h).Str("cmd", c).Send()
_, err := cmd.RunCmdOnHost(cmd.GenerateLogger(opts), opts)
cmd.GenerateLogger(opts)
_, err := cmd.RunCmdOnHost(cmd.cmdLoggers.cmdContxt, opts)
if err != nil {
opts.Logger.Err(err).Str("host", h).Str("cmd", c).Send()
}
+8 -4
View File
@@ -8,11 +8,11 @@ import (
"strings"
)
const _CommandTypeName = "scriptscriptFileremoteScriptpackageuser"
const _CommandTypeName = "scriptscriptFileremoteScriptpackageuserfile"
var _CommandTypeIndex = [...]uint8{0, 0, 6, 16, 28, 35, 39}
var _CommandTypeIndex = [...]uint8{0, 0, 6, 16, 28, 35, 39, 43}
const _CommandTypeLowerName = "scriptscriptfileremotescriptpackageuser"
const _CommandTypeLowerName = "scriptscriptfileremotescriptpackageuserfile"
func (i CommandType) String() string {
if i < 0 || i >= CommandType(len(_CommandTypeIndex)-1) {
@@ -31,9 +31,10 @@ func _CommandTypeNoOp() {
_ = x[RemoteScriptCommandType-(3)]
_ = x[PackageCommandType-(4)]
_ = x[UserCommandType-(5)]
_ = x[FileCommandType-(6)]
}
var _CommandTypeValues = []CommandType{DefaultCommandType, ScriptCommandType, ScriptFileCommandType, RemoteScriptCommandType, PackageCommandType, UserCommandType}
var _CommandTypeValues = []CommandType{DefaultCommandType, ScriptCommandType, ScriptFileCommandType, RemoteScriptCommandType, PackageCommandType, UserCommandType, FileCommandType}
var _CommandTypeNameToValueMap = map[string]CommandType{
_CommandTypeName[0:0]: DefaultCommandType,
@@ -48,6 +49,8 @@ var _CommandTypeNameToValueMap = map[string]CommandType{
_CommandTypeLowerName[28:35]: PackageCommandType,
_CommandTypeName[35:39]: UserCommandType,
_CommandTypeLowerName[35:39]: UserCommandType,
_CommandTypeName[39:43]: FileCommandType,
_CommandTypeLowerName[39:43]: FileCommandType,
}
var _CommandTypeNames = []string{
@@ -57,6 +60,7 @@ var _CommandTypeNames = []string{
_CommandTypeName[16:28],
_CommandTypeName[28:35],
_CommandTypeName[35:39],
_CommandTypeName[39:43],
}
// CommandTypeString retrieves an enum value from the enum constants string name.
-1
View File
@@ -549,7 +549,6 @@ func processCmds(opts *ConfigOpts) error {
}
if !IsHostLocal(cmd.Host) {
cmdHost := replaceVarInString(opts.Vars, cmd.Host, opts.Logger)
if cmdHost != cmd.Host {
cmd.Host = cmdHost
+3 -1
View File
@@ -73,6 +73,8 @@ func (opts *ConfigOpts) Cron() {
srv := server.NewServer(s, opts.GoCron.Port)
// srv := server.NewServer(scheduler, 8080, server.WithTitle("My Custom Scheduler")) // with custom title if you want to customize the title of the UI (optional)
opts.Logger.Info().Msgf("GoCron UI available at http://%s", opts.GoCron.BindAddress)
opts.Logger.Fatal().Msg(http.ListenAndServe(opts.GoCron.BindAddress, srv.Router).Error())
if err := http.ListenAndServe(opts.GoCron.BindAddress, srv.Router); err != nil {
opts.Logger.Fatal().Msg(err.Error())
}
select {} // wait forever
}
+197
View File
@@ -0,0 +1,197 @@
package backy
import (
"fmt"
"io/fs"
"os"
"github.com/pkg/sftp"
"golang.org/x/crypto/ssh"
)
type LocalFileCommandExecutor struct{}
func (f *LocalFileCommandExecutor) Execute(cmd *Command) error {
localExecutor := LocalCommandExecutor{}
switch cmd.FileOperation {
case "copy":
return localExecutor.copyFile(cmd.Source, cmd.Destination, cmd.Permissions)
default:
return fmt.Errorf("unsupported file operation: %s", cmd.FileOperation)
}
}
func (f *LocalFileCommandExecutor) ReadLocalFile(path string) ([]byte, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
return data, nil
}
func (f *LocalFileCommandExecutor) WriteLocalFile(path string, data []byte, Perms fs.FileMode) error {
err := os.WriteFile(path, data, Perms)
if err != nil {
return err
}
return nil
}
func (f *LocalCommandExecutor) copyFile(source, destination string, Perms fs.FileMode) error {
input, err := os.ReadFile(source)
if err != nil {
return err
}
err = os.WriteFile(destination, input, Perms)
if err != nil {
return err
}
return nil
}
type RemoteFileCommandExecutor struct{}
func (r *RemoteFileCommandExecutor) Execute(cmd *Command) error {
remoteExecutor := RemoteFileCommandExecutor{}
sourceTypeLocal := false
if cmd.SourceType == "local" {
sourceTypeLocal = true
}
switch cmd.FileOperation {
case "copy":
return remoteExecutor.copyFile(cmd.Source, cmd.Destination, cmd.Permissions, cmd.RemoteHost.SshClient, sourceTypeLocal)
default:
return fmt.Errorf("unsupported file operation: %s", cmd.FileOperation)
}
}
func (r *RemoteFileCommandExecutor) copyFile(source, destination string, Perms fs.FileMode, sshClient *ssh.Client, sourceTypeLocal bool) error {
if sourceTypeLocal {
input, err := os.ReadFile(source)
if err != nil {
return err
}
sftpClient, sftpErr := sftp.NewClient(sshClient)
if sftpErr != nil {
return sftpErr
}
defer sftpClient.Close()
destFile, err := sftpClient.Create(destination)
if err != nil {
return err
}
defer destFile.Close()
_, err = destFile.Write(input)
if err != nil {
return err
}
err = destFile.Chmod(Perms)
if err != nil {
return err
}
return nil
}
if sshClient == nil {
return fmt.Errorf("SSH Client is nil")
}
client, err := sftp.NewClient(sshClient)
if err != nil {
return err
}
srcFile, err := client.Open(source)
if err != nil {
return err
}
defer srcFile.Close()
destFile, err := client.Create(destination)
if err != nil {
return err
}
defer destFile.Close()
_, err = srcFile.WriteTo(destFile)
if err != nil {
return err
}
err = destFile.Chmod(Perms)
if err != nil {
return err
}
return nil
}
func (r *RemoteFileCommandExecutor) ReadRemoteFile(path string, sshClient *ssh.Client) ([]byte, error) {
sftpClient, sftpErr := sftp.NewClient(sshClient)
if sftpErr != nil {
return nil, sftpErr
}
defer sftpClient.Close()
file, err := sftpClient.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
var fileData []byte
_, err = file.Read(fileData)
if err != nil {
return nil, err
}
return fileData, nil
}
func (r *RemoteFileCommandExecutor) WriteRemoteFile(path string, data []byte, Perms fs.FileMode, sshClient *ssh.Client) error {
sftpClient, sftpErr := sftp.NewClient(sshClient)
if sftpErr != nil {
return sftpErr
}
defer sftpClient.Close()
file, err := sftpClient.Create(path)
if err != nil {
return err
}
defer file.Close()
_, err = file.Write(data)
if err != nil {
return err
}
err = file.Chmod(Perms)
if err != nil {
return err
}
return nil
}
type FileCommandExecutor interface {
Execute(cmd *Command) error
}
func NewFileCommandExecutor(isRemote bool) FileCommandExecutor {
if isRemote {
return &RemoteFileCommandExecutor{}
}
return &LocalFileCommandExecutor{}
}
+78
View File
@@ -0,0 +1,78 @@
package backy
import (
"os"
"testing"
)
func TestCopyFileLocal(t *testing.T) {
FileCommand := Command{}
FileCommand.Type = FileCommandType
FileCommand.FileOperation = "copy"
FileCommand.Permissions = 0644
FileCommand.Source = "/home/andrew/Projects/backy/tests/data/fileops/source.txt"
FileCommand.Destination = "/home/andrew/Projects/backy/tests/data/fileops/destination.txt"
var FileCommandExecutor = LocalFileCommandExecutor{}
err := FileCommandExecutor.Execute(&FileCommand)
if err != nil {
t.Errorf("Error executing file command: %v", err)
}
srcBytes, srcErr := os.ReadFile(FileCommand.Source)
if srcErr != nil {
t.Errorf("Error reading source file: %v", srcErr)
}
destBytes, destErr := os.ReadFile(FileCommand.Destination)
if destErr != nil {
t.Errorf("Error reading destination file: %v", destErr)
}
if string(srcBytes) != string(destBytes) {
t.Errorf("Source and destination files do not match")
}
// Additional checks can be added here to verify the file was copied correctly
}
func TestCopyFileRemote(t *testing.T) {
opts := NewConfigOptions("")
opts.Hosts = map[string]*Host{}
RemoteFileCommand := Command{}
RemoteFileCommand.Type = FileCommandType
RemoteFileCommand.FileOperation = "copy"
RemoteFileCommand.Permissions = 0644
RemoteFileCommand.Destination = "/home/backy/destination.txt"
RemoteFileCommand.Source = "/home/andrew/Projects/backy/tests/data/fileops/source.txt"
RemoteFileCommand.Host = "localhost"
RemoteFileCommand.RemoteHost = &Host{
HostName: "localhost",
User: "backy",
Password: "backy",
Port: 2222,
PrivateKeyPath: "/home/andrew/Projects/backy/tests/docker/backytest",
KnownHostsFile: "/home/andrew/Projects/backy/tests/docker/known_hosts",
}
sshErr := RemoteFileCommand.RemoteHost.ConnectToHost(opts)
if sshErr != nil {
t.Errorf("Error connecting to remote host: %v", sshErr)
}
var RemoteFileCommandExecutor = RemoteFileCommandExecutor{}
err := RemoteFileCommandExecutor.Execute(&RemoteFileCommand)
if err != nil {
t.Errorf("Error executing remote file command: %v", err)
}
srcBytes, srcErr := os.ReadFile(RemoteFileCommand.Source)
if srcErr != nil {
t.Errorf("Error reading source file: %v", srcErr)
}
destBytes, destErr := RemoteFileCommandExecutor.ReadRemoteFile(RemoteFileCommand.Destination, RemoteFileCommand.RemoteHost.SshClient)
if destErr != nil {
t.Errorf("Error reading destination file: %v", destErr)
}
if string(srcBytes) != string(destBytes) {
t.Errorf("Source and destination files do not match")
}
// Additional checks can be added here to verify the file was copied correctly
}
+141
View File
@@ -0,0 +1,141 @@
// Code generated by "enumer -linecomment -yaml -text -json -type=FileCommandOperation"; DO NOT EDIT.
package backy
import (
"encoding/json"
"fmt"
"strings"
)
const _FileCommandOperationName = "copymovedeletechownchmod"
var _FileCommandOperationIndex = [...]uint8{0, 0, 4, 8, 14, 19, 24}
const _FileCommandOperationLowerName = "copymovedeletechownchmod"
func (i FileCommandOperation) String() string {
if i < 0 || i >= FileCommandOperation(len(_FileCommandOperationIndex)-1) {
return fmt.Sprintf("FileCommandOperation(%d)", i)
}
return _FileCommandOperationName[_FileCommandOperationIndex[i]:_FileCommandOperationIndex[i+1]]
}
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
func _FileCommandOperationNoOp() {
var x [1]struct{}
_ = x[DefaultFCO-(0)]
_ = x[FileCommandOperationCopy-(1)]
_ = x[FileCommandOperationMove-(2)]
_ = x[FileCommandOperationDelete-(3)]
_ = x[FileCommandOperationChown-(4)]
_ = x[FileCommandOperationChmod-(5)]
}
var _FileCommandOperationValues = []FileCommandOperation{DefaultFCO, FileCommandOperationCopy, FileCommandOperationMove, FileCommandOperationDelete, FileCommandOperationChown, FileCommandOperationChmod}
var _FileCommandOperationNameToValueMap = map[string]FileCommandOperation{
_FileCommandOperationName[0:0]: DefaultFCO,
_FileCommandOperationLowerName[0:0]: DefaultFCO,
_FileCommandOperationName[0:4]: FileCommandOperationCopy,
_FileCommandOperationLowerName[0:4]: FileCommandOperationCopy,
_FileCommandOperationName[4:8]: FileCommandOperationMove,
_FileCommandOperationLowerName[4:8]: FileCommandOperationMove,
_FileCommandOperationName[8:14]: FileCommandOperationDelete,
_FileCommandOperationLowerName[8:14]: FileCommandOperationDelete,
_FileCommandOperationName[14:19]: FileCommandOperationChown,
_FileCommandOperationLowerName[14:19]: FileCommandOperationChown,
_FileCommandOperationName[19:24]: FileCommandOperationChmod,
_FileCommandOperationLowerName[19:24]: FileCommandOperationChmod,
}
var _FileCommandOperationNames = []string{
_FileCommandOperationName[0:0],
_FileCommandOperationName[0:4],
_FileCommandOperationName[4:8],
_FileCommandOperationName[8:14],
_FileCommandOperationName[14:19],
_FileCommandOperationName[19:24],
}
// FileCommandOperationString retrieves an enum value from the enum constants string name.
// Throws an error if the param is not part of the enum.
func FileCommandOperationString(s string) (FileCommandOperation, error) {
if val, ok := _FileCommandOperationNameToValueMap[s]; ok {
return val, nil
}
if val, ok := _FileCommandOperationNameToValueMap[strings.ToLower(s)]; ok {
return val, nil
}
return 0, fmt.Errorf("%s does not belong to FileCommandOperation values", s)
}
// FileCommandOperationValues returns all values of the enum
func FileCommandOperationValues() []FileCommandOperation {
return _FileCommandOperationValues
}
// FileCommandOperationStrings returns a slice of all String values of the enum
func FileCommandOperationStrings() []string {
strs := make([]string, len(_FileCommandOperationNames))
copy(strs, _FileCommandOperationNames)
return strs
}
// IsAFileCommandOperation returns "true" if the value is listed in the enum definition. "false" otherwise
func (i FileCommandOperation) IsAFileCommandOperation() bool {
for _, v := range _FileCommandOperationValues {
if i == v {
return true
}
}
return false
}
// MarshalJSON implements the json.Marshaler interface for FileCommandOperation
func (i FileCommandOperation) MarshalJSON() ([]byte, error) {
return json.Marshal(i.String())
}
// UnmarshalJSON implements the json.Unmarshaler interface for FileCommandOperation
func (i *FileCommandOperation) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return fmt.Errorf("FileCommandOperation should be a string, got %s", data)
}
var err error
*i, err = FileCommandOperationString(s)
return err
}
// MarshalText implements the encoding.TextMarshaler interface for FileCommandOperation
func (i FileCommandOperation) MarshalText() ([]byte, error) {
return []byte(i.String()), nil
}
// UnmarshalText implements the encoding.TextUnmarshaler interface for FileCommandOperation
func (i *FileCommandOperation) UnmarshalText(text []byte) error {
var err error
*i, err = FileCommandOperationString(string(text))
return err
}
// MarshalYAML implements a YAML Marshaler for FileCommandOperation
func (i FileCommandOperation) MarshalYAML() (interface{}, error) {
return i.String(), nil
}
// UnmarshalYAML implements a YAML Unmarshaler for FileCommandOperation
func (i *FileCommandOperation) UnmarshalYAML(unmarshal func(interface{}) error) error {
var s string
if err := unmarshal(&s); err != nil {
return err
}
var err error
*i, err = FileCommandOperationString(s)
return err
}
+50 -12
View File
@@ -36,6 +36,8 @@ func (opts *ConfigOpts) ListCommand(cmd string) {
// print the command's information
if cmdFound {
print("Backy Command: ")
println(cmd)
println("Command: ")
print(cmdInfo.Cmd)
@@ -45,17 +47,57 @@ func (opts *ConfigOpts) ListCommand(cmd string) {
print(v) // print command arg
}
// is it remote or local
if !IsHostLocal(cmdInfo.Host) {
if cmdInfo.Type.String() != "" {
println()
print("Host: ", cmdInfo.Host)
print("Type: ", cmdInfo.Type.String())
println()
}
if cmdInfo.Hosts != nil {
for n, h := range cmdInfo.Hosts {
println()
fmt.Printf("Host %d: %s", n, h)
println()
}
} else {
println()
print("Host: Runs on Local Machine\n\n")
// is it remote or local
if !IsHostLocal(cmdInfo.Host) {
println()
print("Host: ", cmdInfo.Host)
println()
} else {
println()
print("Host: Runs on Local Machine\n\n")
}
}
if len(cmdInfo.Environment) > 0 {
fmt.Print("Environment: ")
for _, env := range cmdInfo.Environment {
fmt.Printf("%s ", env)
}
fmt.Println()
}
if cmdInfo.PackageManager != "" {
fmt.Printf("\nPackage Manager: %s\n", cmdInfo.PackageManager)
if len(cmdInfo.Packages) > 0 {
for _, pkg := range cmdInfo.Packages {
fmt.Printf("Package: %v\n", pkg)
}
}
}
if cmdInfo.Username != "" || cmdInfo.UserID != "" {
fmt.Println("\nUser Settings:")
fmt.Printf("Username: %s\n", cmdInfo.Username)
fmt.Printf("UserID: %s\n", cmdInfo.UserID)
}
if cmdInfo.Dir != nil {
@@ -64,11 +106,6 @@ func (opts *ConfigOpts) ListCommand(cmd string) {
println()
}
if cmdInfo.Type.String() != "" {
print("Type: ", cmdInfo.Type.String())
println()
}
} else {
fmt.Printf("Command %s not found. Check spelling.\n", cmd)
@@ -100,9 +137,10 @@ func (opts *ConfigOpts) ListCommandList(list string) {
println("List: ", list)
println()
for _, v := range listInfo.Order {
for n, cmd := range listInfo.Order {
println()
opts.ListCommand(v)
fmt.Printf("Command %d: \n", n+1)
opts.ListCommand(cmd)
}
} else {
+12
View File
@@ -2,6 +2,7 @@ package backy
import (
"encoding/json"
"fmt"
"os"
"time"
)
@@ -75,6 +76,17 @@ func (metricFile *MetricFile) SaveToFile() error {
}
func LoadMetricsFromFile(filename string) (*MetricFile, error) {
if err := testFile(filename); err != nil {
_, createErr := os.Create(filename)
if createErr != nil {
return nil, fmt.Errorf("error creating file and root cause: %w", err)
}
metricData := NewMetricsFromFile(filename)
data, _ := json.MarshalIndent(metricData, "", " ")
os.WriteFile(filename, data, 0600)
return metricData, nil
}
jsonData, err := os.ReadFile(filename)
if err != nil {
return nil, err
+57 -62
View File
@@ -1,67 +1,62 @@
package backy
// import (
// "testing"
// "time"
// )
import (
"testing"
"time"
)
// func TestAddingMetricsForCommand(t *testing.T) {
func TestAddingMetricsForCommand(t *testing.T) {
// Create a new MetricFile
metricFile := NewMetricsFromFile("test_metrics.json")
metricFile, err := LoadMetricsFromFile(metricFile.Filename)
if err != nil {
t.Fatalf("Failed to load metrics from file: %v", err)
}
// Add metrics for a command
commandName := "test_command"
if metricFile.CommandMetrics != nil {
if _, exists := metricFile.CommandMetrics[commandName]; !exists {
metricFile.CommandMetrics[commandName] = NewMetrics()
}
}
// Update the metrics for the command
executionTime := 1.8 // Example execution time in seconds
success := true // Example success status
metricFile.CommandMetrics[commandName].Update(success, executionTime, time.Now())
// Check if the metrics were updated correctly
if metricFile.CommandMetrics[commandName].SuccessfulExecutions > 50 {
t.Errorf("Expected 1 successful execution, got %d", metricFile.CommandMetrics[commandName].SuccessfulExecutions)
}
if metricFile.CommandMetrics[commandName].TotalExecutions > 50 {
t.Errorf("Expected 1 total execution, got %d", metricFile.CommandMetrics[commandName].TotalExecutions)
}
if metricFile.CommandMetrics[commandName].TotalExecutionTime != executionTime {
t.Errorf("Expected execution time %f, got %f", executionTime, metricFile.CommandMetrics[commandName].TotalExecutionTime)
}
err = metricFile.SaveToFile()
if err != nil {
t.Errorf("Failed to save metrics to file: %v", err)
}
listName := "test_list"
if _, exists := metricFile.ListMetrics[listName]; !exists {
metricFile.ListMetrics[listName] = NewMetrics()
}
// Update the metrics for the list
metricFile.ListMetrics[listName].Update(success, executionTime, time.Now())
if metricFile.ListMetrics[listName].SuccessfulExecutions > 50 {
t.Errorf("Expected 1 successful execution for list, got %d", metricFile.ListMetrics[listName].SuccessfulExecutions)
}
if metricFile.ListMetrics[listName].TotalExecutions > 50 {
t.Errorf("Expected 1 total execution for list, got %d", metricFile.ListMetrics[listName].TotalExecutions)
}
if metricFile.ListMetrics[listName].TotalExecutionTime > executionTime {
t.Errorf("Expected execution time %f for list, got %f", executionTime, metricFile.ListMetrics[listName].TotalExecutionTime)
}
// // Create a new MetricFile
// metricFile := NewMetricsFromFile("test_metrics.json")
// Save the metrics to a file
err = metricFile.SaveToFile()
if err != nil {
t.Errorf("Failed to save metrics to file: %v", err)
}
// metricFile, err := LoadMetricsFromFile(metricFile.Filename)
// if err != nil {
// t.Errorf("Failed to load metrics from file: %v", err)
// }
// // Add metrics for a command
// commandName := "test_command"
// if _, exists := metricFile.CommandMetrics[commandName]; !exists {
// metricFile.CommandMetrics[commandName] = NewMetrics()
// }
// // Update the metrics for the command
// executionTime := 1.8 // Example execution time in seconds
// success := true // Example success status
// metricFile.CommandMetrics[commandName].Update(success, executionTime, time.Now())
// // Check if the metrics were updated correctly
// if metricFile.CommandMetrics[commandName].SuccessfulExecutions > 50 {
// t.Errorf("Expected 1 successful execution, got %d", metricFile.CommandMetrics[commandName].SuccessfulExecutions)
// }
// if metricFile.CommandMetrics[commandName].TotalExecutions > 50 {
// t.Errorf("Expected 1 total execution, got %d", metricFile.CommandMetrics[commandName].TotalExecutions)
// }
// // if metricFile.CommandMetrics[commandName].TotalExecutionTime != executionTime {
// // t.Errorf("Expected execution time %f, got %f", executionTime, metricFile.CommandMetrics[commandName].TotalExecutionTime)
// // }
// err = metricFile.SaveToFile()
// if err != nil {
// t.Errorf("Failed to save metrics to file: %v", err)
// }
// listName := "test_list"
// if _, exists := metricFile.ListMetrics[listName]; !exists {
// metricFile.ListMetrics[listName] = NewMetrics()
// }
// // Update the metrics for the list
// metricFile.ListMetrics[listName].Update(success, executionTime, time.Now())
// if metricFile.ListMetrics[listName].SuccessfulExecutions > 50 {
// t.Errorf("Expected 1 successful execution for list, got %d", metricFile.ListMetrics[listName].SuccessfulExecutions)
// }
// if metricFile.ListMetrics[listName].TotalExecutions > 50 {
// t.Errorf("Expected 1 total execution for list, got %d", metricFile.ListMetrics[listName].TotalExecutions)
// }
// // if metricFile.ListMetrics[listName].TotalExecutionTime > executionTime {
// // t.Errorf("Expected execution time %f for list, got %f", executionTime, metricFile.ListMetrics[listName].TotalExecutionTime)
// // }
// // Save the metrics to a file
// err = metricFile.SaveToFile()
// if err != nil {
// t.Errorf("Failed to save metrics to file: %v", err)
// }
// }
}
+64 -50
View File
@@ -5,12 +5,12 @@
package backy
import (
"bufio"
"bytes"
"fmt"
"io"
"os"
"os/user"
"path/filepath"
"strconv"
"strings"
"time"
@@ -472,10 +472,31 @@ func (command *Command) RunCmdOnHost(cmdCtxLogger zerolog.Logger, opts *ConfigOp
defer commandSession.Close()
// Set output writers
cmdOutWriters = io.MultiWriter(&cmdOutBuf)
if IsCmdStdOutEnabled() {
cmdOutWriters = io.MultiWriter(os.Stdout, &cmdOutBuf)
// var file *os.File
if !IsHostLocal(command.Host) && command.Output.File != "" {
if filepath.IsAbs(command.Output.File) {
fileName := filepath.Base(command.Output.File)
fileName = fmt.Sprintf("%s_%s", command.RemoteHost.Host, fileName)
command.Output.File = filepath.Join(filepath.Dir(command.Output.File), fileName)
} else {
command.Output.File = fmt.Sprintf("%s_%s", command.RemoteHost.Host, command.Output.File)
}
}
cmdOutWriters, _, err = makeCmdOutWriters(&cmdOutBuf, command.Output.File)
if err != nil {
return nil, fmt.Errorf("error creating command output writers: %w", err)
}
// defer func() {
// if file != nil {
// file.Close()
// }
// }()
// cmdOutWriters = logging.SetLoggingWriterForCommand(&cmdOutBuf, command.Output.File, IsCmdStdOutEnabled())
cmdCtxLogger = zerolog.New(cmdOutWriters).With().Timestamp().Logger()
// cmdCtxLogger = command.GenerateLoggerForCmd(cmdCtxLogger)
command.cmdLoggers.cmdContxt = cmdCtxLogger
// cmdCtxLogger.Info().Msgf("Executing %s", command.Cmd)
commandSession.Stdout = cmdOutWriters
commandSession.Stderr = cmdOutWriters
@@ -492,11 +513,13 @@ func (command *Command) RunCmdOnHost(cmdCtxLogger zerolog.Logger, opts *ConfigOp
// Handle command execution based on type
switch command.Type {
case ScriptCommandType:
return command.runScript(commandSession, cmdCtxLogger, &cmdOutBuf)
return command.runScript(commandSession, &cmdOutBuf)
case RemoteScriptCommandType:
return command.runRemoteScript(commandSession, cmdCtxLogger, &cmdOutBuf)
return command.runRemoteScript(commandSession, &cmdOutBuf)
case ScriptFileCommandType:
return command.runScriptFile(commandSession, cmdCtxLogger, &cmdOutBuf)
// commandSession.Stdout = nil
// commandSession.Stderr = nil
return command.runScriptFile(commandSession, &cmdOutBuf)
case PackageCommandType:
var remoteHostPackageExecutor RemoteHostPackageExecutor
return remoteHostPackageExecutor.RunCmdOnHost(command, commandSession, cmdCtxLogger, cmdOutBuf)
@@ -518,18 +541,18 @@ func (command *Command) RunCmdOnHost(cmdCtxLogger zerolog.Logger, opts *ConfigOp
userNamePass := fmt.Sprintf("%s:%s", command.Username, command.UserPassword)
client, err := sftp.NewClient(command.RemoteHost.SshClient)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error creating sftp client: %v", err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error creating sftp client: %v", err)
}
uuidFile := uuid.New()
passFilePath := fmt.Sprintf("/tmp/%s", uuidFile.String())
passFile, passFileErr := client.Create(passFilePath)
if passFileErr != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error creating file /tmp/%s: %v", uuidFile.String(), passFileErr)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error creating file /tmp/%s: %v", uuidFile.String(), passFileErr)
}
_, err = passFile.Write([]byte(userNamePass))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error writing to file /tmp/%s: %v", uuidFile.String(), err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error writing to file /tmp/%s: %v", uuidFile.String(), err)
}
ArgsStr = fmt.Sprintf("cat %s | chpasswd", passFilePath)
@@ -543,7 +566,7 @@ func (command *Command) RunCmdOnHost(cmdCtxLogger zerolog.Logger, opts *ConfigOp
defer rmFileFunc()
}
if err := commandSession.Run(command.ArgStr); err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error running command: %w", err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error running command: %w", err)
}
if command.Type == UserCommandType {
@@ -564,41 +587,41 @@ func (command *Command) RunCmdOnHost(cmdCtxLogger zerolog.Logger, opts *ConfigOp
commandSession, _ = command.RemoteHost.createSSHSession(opts)
userHome, err = commandSession.CombinedOutput(fmt.Sprintf("grep \"%s\" /etc/passwd | cut -d: -f6", command.Username))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error finding user home from /etc/passwd: %v", err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error finding user home from /etc/passwd: %v", err)
}
command.UserHome = strings.TrimSpace(string(userHome))
userSshDir := fmt.Sprintf("%s/.ssh", command.UserHome)
client, err = sftp.NewClient(command.RemoteHost.SshClient)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error creating sftp client: %v", err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error creating sftp client: %v", err)
}
err = client.MkdirAll(userSshDir)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error creating directory %s: %v", userSshDir, err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error creating directory %s: %v", userSshDir, err)
}
_, err = client.Create(fmt.Sprintf("%s/authorized_keys", userSshDir))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
}
f, err = client.OpenFile(fmt.Sprintf("%s/authorized_keys", userSshDir), os.O_APPEND|os.O_CREATE|os.O_WRONLY)
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error opening file %s/authorized_keys: %v", userSshDir, err)
}
defer f.Close()
for _, k := range command.UserSshPubKeys {
buf := bytes.NewBufferString(k)
cmdCtxLogger.Info().Str("key", k).Msg("adding SSH key")
if _, err := f.ReadFrom(buf); err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error adding to authorized keys: %v", err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error adding to authorized keys: %v", err)
}
}
commandSession, _ = command.RemoteHost.createSSHSession(opts)
_, err = commandSession.CombinedOutput(fmt.Sprintf("chown -R %s:%s %s", command.Username, command.Username, userHome))
if err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), err
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), err
}
}
@@ -606,7 +629,7 @@ func (command *Command) RunCmdOnHost(cmdCtxLogger zerolog.Logger, opts *ConfigOp
}
}
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), nil
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), nil
}
func checkPackageVersion(cmdCtxLogger zerolog.Logger, command *Command, commandSession *ssh.Session, cmdOutBuf bytes.Buffer) ([]string, error) {
@@ -626,9 +649,9 @@ func checkPackageVersion(cmdCtxLogger zerolog.Logger, command *Command, commandS
_, parseErr := parsePackageVersion(string(cmdOut), cmdCtxLogger, command, cmdOutBuf)
if parseErr != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error: packages %v not listed: %w", command.Packages, err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error: packages %v not listed: %w", command.Packages, err)
}
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error running %s: %w", ArgsStr, err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error running %s: %w", ArgsStr, err)
}
return parsePackageVersion(string(cmdOut), cmdCtxLogger, command, cmdOutBuf)
@@ -643,7 +666,7 @@ func getCommandTypeAndSetCommandInfoLabel(commandType CommandType) string {
}
// runScript handles the execution of inline scripts.
func (command *Command) runScript(session *ssh.Session, cmdCtxLogger zerolog.Logger, outputBuf *bytes.Buffer) ([]string, error) {
func (command *Command) runScript(session *ssh.Session, outputBuf *bytes.Buffer) ([]string, error) {
script, err := command.prepareScriptBuffer()
if err != nil {
return nil, err
@@ -655,14 +678,14 @@ func (command *Command) runScript(session *ssh.Session, cmdCtxLogger zerolog.Log
}
if err := session.Wait(); err != nil {
return collectOutput(outputBuf, command.Name, cmdCtxLogger, true), fmt.Errorf("error waiting for shell: %w", err)
return collectOutput(outputBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error waiting for command: %w", err)
}
return collectOutput(outputBuf, command.Name, cmdCtxLogger, command.Output.ToLog), nil
return collectOutput(outputBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), nil
}
// runScriptFile handles the execution of script files.
func (command *Command) runScriptFile(session *ssh.Session, cmdCtxLogger zerolog.Logger, outputBuf *bytes.Buffer) ([]string, error) {
func (command *Command) runScriptFile(session *ssh.Session, outputBuf *bytes.Buffer) ([]string, error) {
script, err := command.prepareScriptFileBuffer()
if err != nil {
return nil, err
@@ -674,10 +697,10 @@ func (command *Command) runScriptFile(session *ssh.Session, cmdCtxLogger zerolog
}
if err := session.Wait(); err != nil {
return collectOutput(outputBuf, command.Name, cmdCtxLogger, true), fmt.Errorf("error waiting for shell: %w", err)
return collectOutput(outputBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error waiting for shell: %w", err)
}
return collectOutput(outputBuf, command.Name, cmdCtxLogger, command.Output.ToLog), nil
return collectOutput(outputBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), nil
}
// prepareScriptBuffer prepares a buffer for inline scripts.
@@ -685,7 +708,7 @@ func (command *Command) prepareScriptBuffer() (*bytes.Buffer, error) {
var buffer bytes.Buffer
for _, envVar := range command.Environment {
buffer.WriteString(fmt.Sprintf("export %s", envVar))
fmt.Fprintf(&buffer, "export %s", envVar)
buffer.WriteByte('\n')
}
@@ -700,7 +723,8 @@ func (command *Command) prepareScriptBuffer() (*bytes.Buffer, error) {
buffer.WriteString(command.Cmd)
for _, arg := range command.Args {
buffer.WriteString(" " + arg)
buffer.WriteString(" ")
buffer.WriteString(arg)
}
buffer.WriteByte('\n')
return &buffer, nil
@@ -710,8 +734,12 @@ func (command *Command) prepareScriptBuffer() (*bytes.Buffer, error) {
func (command *Command) prepareScriptFileBuffer() (*bytes.Buffer, error) {
var buffer bytes.Buffer
// if !command.SaveShellHistory {
// buffer.WriteString("unset HISTFILE\nexport HISTSIZE=0\nexport SAVEHIST=0\n")
// }
for _, envVar := range command.Environment {
buffer.WriteString(fmt.Sprintf("export %s", envVar))
fmt.Fprintf(&buffer, "export %s", envVar)
buffer.WriteByte('\n')
}
@@ -736,7 +764,7 @@ func (command *Command) prepareScriptFileBuffer() (*bytes.Buffer, error) {
}
// runRemoteScript handles the execution of remote scripts
func (command *Command) runRemoteScript(session *ssh.Session, cmdCtxLogger zerolog.Logger, outputBuf *bytes.Buffer) ([]string, error) {
func (command *Command) runRemoteScript(session *ssh.Session, outputBuf *bytes.Buffer) ([]string, error) {
script, err := command.Fetcher.Fetch(command.Cmd)
if err != nil {
return nil, err
@@ -748,10 +776,10 @@ func (command *Command) runRemoteScript(session *ssh.Session, cmdCtxLogger zerol
err = session.Run(command.Shell)
if err != nil {
return collectOutput(outputBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error running remote script: %w", err)
return collectOutput(outputBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error running remote script: %w", err)
}
return collectOutput(outputBuf, command.Name, cmdCtxLogger, command.Output.ToLog), nil
return collectOutput(outputBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), nil
}
// readFileToBuffer reads a file into a buffer.
@@ -774,20 +802,6 @@ func readFileToBuffer(filePath string) (*bytes.Buffer, error) {
return &buffer, nil
}
// collectOutput collects output from a buffer and logs it.
func collectOutput(buf *bytes.Buffer, commandName string, logger zerolog.Logger, wantOutput bool) []string {
var outputArr []string
scanner := bufio.NewScanner(buf)
for scanner.Scan() {
line := scanner.Text()
outputArr = append(outputArr, line)
if wantOutput {
logger.Info().Str("cmd", commandName).Str("output", line).Send()
}
}
return outputArr
}
// createSSHSession attempts to create a new SSH session and retries on failure.
func (h *Host) createSSHSession(opts *ConfigOpts) (*ssh.Session, error) {
session, err := h.SshClient.NewSession()
@@ -834,7 +848,7 @@ func DoesHostHaveHostName(host string) (bool, string) {
}
func IsHostLocal(host string) bool {
host = strings.ToLower(host)
host = strings.ToLower(strings.TrimSpace(host))
return host == "127.0.0.1" || host == "localhost" || host == ""
}
@@ -864,7 +878,7 @@ func (r RemoteHostPackageExecutor) RunCmdOnHost(command *Command, commandSession
cmdCtxLogger.Debug().Str("cmd + args", ArgsStr).Send()
// Run simple command
if err := commandSession.Run(ArgsStr); err != nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error running command: %w", err)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error running command: %w", err)
}
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), nil
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), nil
}
+71
View File
@@ -0,0 +1,71 @@
package backy
import (
"bytes"
"os"
"path/filepath"
"text/template"
"gopkg.in/yaml.v3"
)
func LoadVarsYAML(path string) (map[string]interface{}, error) {
b, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var m map[string]interface{}
if err := yaml.Unmarshal(b, &m); err != nil {
return nil, err
}
return m, nil
}
func RenderTemplateFile(templatePath string, vars map[string]interface{}) ([]byte, error) {
tmplText, err := os.ReadFile(templatePath)
if err != nil {
return nil, err
}
funcs := template.FuncMap{
"env": func(k, d string) string {
if v := os.Getenv(k); v != "" {
return v
}
return d
},
"default": func(def interface{}, v interface{}) interface{} {
if v == nil {
return def
}
if s, ok := v.(string); ok && s == "" {
return def
}
return v
},
"toYaml": func(v interface{}) string {
b, _ := yaml.Marshal(v)
return string(b)
},
}
t := template.New(filepath.Base(templatePath)).Funcs(funcs)
t, err = t.Parse(string(tmplText))
if err != nil {
return nil, err
}
var buf bytes.Buffer
if err := t.Execute(&buf, vars); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
func WriteRenderedFile(templatePath string, vars map[string]interface{}, dest string, perm os.FileMode) error {
out, err := RenderTemplateFile(templatePath, vars)
if err != nil {
return err
}
if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil {
return err
}
return os.WriteFile(dest, out, perm)
}
+5 -7
View File
@@ -1,15 +1,13 @@
Command list {{.listName }} failed.
Backy Command: {{.CmdName}}.
The command run was {{.CmdName}}.
The command executed: {{.Command}} {{ if .Args }} {{- range .Args}} {{.}} {{end}} {{end}}
The command executed was {{.Command}} {{ if .Args }} {{- range .Args}} {{.}} {{end}} {{end}}
{{ if .Err }} Error: {{ .Err }}{{ end }}
{{ if .Err }} The error was {{ .Err }}{{ end }}
{{ if .Output }} The output was: {{- range .Output}} {{.}} {{end}} {{end}}
{{ if .Output }} Output: {{- range .Output}} {{.}} {{end}} {{end}}
{{ if .CmdsRan }}
The following commands ran:
Commands executed (last one failed):
{{- range .CmdsRan}}
- {{. -}}
{{end}}
+1 -3
View File
@@ -1,6 +1,4 @@
Command list {{ .listName }} completed successfully.
The following commands ran:
Commands executed:
{{- range .CmdsRan}}
- {{. -}}
{{end}}
+35 -1
View File
@@ -2,6 +2,7 @@ package backy
import (
"bytes"
"io/fs"
"text/template"
"strings"
@@ -57,6 +58,8 @@ type (
// See CommandType enum further down the page for acceptable values
Type CommandType `yaml:"type,omitempty"`
RawOutput bytes.Buffer
Host string `yaml:"host,omitempty"`
Hosts []string `yaml:"hosts,omitempty"`
@@ -138,7 +141,25 @@ type (
// stdin only for userOperation = password (for now)
stdin *strings.Reader
// END USER STRUCommandType FIELDS
// END USER CommandType FIELDS
cmdLoggers struct {
global zerolog.Logger
cmdContxt zerolog.Logger
}
// BEGIN FILE COMMAND FIELDS
FileOperation string `yaml:"fileOperation,omitempty"`
Source string `yaml:"source,omitempty"`
DestinationType string `yaml:"destinationType,omitempty"`
SourceType string `yaml:"sourceType,omitempty"`
Destination string `yaml:"destination,omitempty"`
Permissions fs.FileMode `yaml:"permissions,omitempty"`
Owner string `yaml:"owner,omitempty"`
Group string `yaml:"group,omitempty"`
// END FILE COMMAND FIELDS
}
RemoteSource struct {
@@ -289,6 +310,7 @@ type (
}
CmdResult struct {
HostName string
CmdName string // Name of the command executed
ListName string // Name of the command list
Error error // Error encountered, if any
@@ -312,6 +334,7 @@ type (
CommandType int
PackageOperation int
AllowedExternalDirectives int
FileCommandOperation int
)
//go:generate go run github.com/dmarkham/enumer -linecomment -yaml -text -json -type=CommandType
@@ -322,6 +345,7 @@ const (
RemoteScriptCommandType // remoteScript
PackageCommandType // package
UserCommandType // user
FileCommandType // file
)
//go:generate go run github.com/dmarkham/enumer -linecomment -yaml -text -json -type=PackageOperation
@@ -335,6 +359,16 @@ const (
PackageOperationIsInstalled // isInstalled
)
//go:generate go run github.com/dmarkham/enumer -linecomment -yaml -text -json -type=FileCommandOperation
const (
DefaultFCO FileCommandOperation = iota //
FileCommandOperationCopy // copy
FileCommandOperationMove // move
FileCommandOperationDelete // delete
FileCommandOperationChown // chown
FileCommandOperationChmod // chmod
)
//go:generate go run github.com/dmarkham/enumer -linecomment -yaml -text -json -type=AllowedExternalDirectives
const (
DefaultExternalDir AllowedExternalDirectives = iota
+66 -8
View File
@@ -5,6 +5,7 @@
package backy
import (
"bufio"
"bytes"
"context"
"errors"
@@ -15,6 +16,7 @@ import (
"path/filepath"
"regexp"
"strings"
"time"
"git.andrewnw.xyz/CyberShell/backy/pkg/logging"
"git.andrewnw.xyz/CyberShell/backy/pkg/remotefetcher"
@@ -173,7 +175,7 @@ errEnvFile:
}
func prependEnvVarsToCommand(envVars environmentVars, opts *ConfigOpts, command string, args []string, cmdCtxLogger zerolog.Logger) string {
var envPrefix string
var envPrefix strings.Builder
if envVars.file != "" {
envPath, envPathErr := getFullPathWithHomeDir(envVars.file)
if envPathErr != nil {
@@ -190,15 +192,15 @@ func prependEnvVarsToCommand(envVars environmentVars, opts *ConfigOpts, command
log.Fatal().Str("envFile", envPath).Err(err).Send()
}
for key, val := range envMap {
envPrefix += fmt.Sprintf("%s=%s ", key, getExternalConfigDirectiveValue(val, opts, AllowedExternalDirectiveVaultEnv))
fmt.Fprintf(&envPrefix, "%s=%s ", key, getExternalConfigDirectiveValue(val, opts, AllowedExternalDirectiveVaultEnv))
}
}
for _, value := range envVars.env {
envVarArr := strings.Split(value, "=")
envPrefix += fmt.Sprintf("%s=%s ", envVarArr[0], getExternalConfigDirectiveValue(envVarArr[1], opts, AllowedExternalDirectiveVault))
envPrefix += "\n"
fmt.Fprintf(&envPrefix, "%s=%s ", envVarArr[0], getExternalConfigDirectiveValue(envVarArr[1], opts, AllowedExternalDirectiveVault))
envPrefix.WriteString("\n")
}
return envPrefix + command + " " + strings.Join(args, " ")
return envPrefix.String() + command + " " + strings.Join(args, " ")
}
func contains(s []string, e string) bool {
@@ -220,6 +222,62 @@ func CheckConfigValues(config *koanf.Koanf, file string) {
}
}
// collectOutput collects output from a buffer and logs it.
func collectOutput(buf *bytes.Buffer, commandName string, logger, globalLogger zerolog.Logger, wantOutput bool) []string {
var outputArr []string
copyBuf := bytes.NewBuffer(buf.Bytes())
if buf.Len() == 0 {
logger.Info().Str("cmd", commandName).Msg("NO OUTPUT!!!!!!!")
globalLogger.Info().Str("cmd", commandName).Msg("NO OUTPUT!!!!!!!")
return outputArr
}
scanner := bufio.NewScanner(copyBuf)
for scanner.Scan() {
line := scanner.Text()
clean := sanitizeString(line)
outputArr = append(outputArr, clean)
if wantOutput {
time.Sleep(10 * time.Millisecond)
logger.Info().Str("cmd", commandName).Str("output", clean).Send()
if IsCmdStdOutEnabled() {
globalLogger.Info().Str("cmd", commandName).Str("output", clean).Send()
}
}
}
buf.Reset()
return outputArr
}
// sanitizeString removes ANSI escape sequences and non-printable control characters
// while preserving tabs. This helps remove color codes and other terminal control
// characters from remote command output.
func sanitizeString(s string) string {
// Remove common ANSI CSI sequences like "\x1b[31m" etc.
var ansiCSI = regexp.MustCompile(`\x1b\[[0-9;?]*[ -/]*[@-~]`)
s = ansiCSI.ReplaceAllString(s, "")
// Remove OSC sequences started by ESC ] and terminated by BEL or ESC\
var osc = regexp.MustCompile(`(?s)"].*?(?:|\\)`)
s = osc.ReplaceAllString(s, "")
// Sometimes the ESC has been stripped earlier and we are left with sequences like "]2;title]1;"
// Remove leftover bracketed sequences like "]<digits>;<text>"
var leftoverOSC = regexp.MustCompile(`\][0-9]+[^\]]*`)
s = leftoverOSC.ReplaceAllString(s, "")
var b strings.Builder
for _, r := range s {
if r == '\t' {
b.WriteString(" ")
}
if r >= 0x20 && r != 0x7f {
b.WriteRune(r)
}
}
return b.String()
}
func testFile(c string) error {
if strings.TrimSpace(c) != "" {
file, fileOpenErr := os.Open(c)
@@ -368,7 +426,7 @@ func parsePackageVersion(output string, cmdCtxLogger zerolog.Logger, command *Co
pkgVersionOnSystem, err := command.pkgMan.ParseRemotePackageManagerVersionOutput(output)
if err != nil {
cmdCtxLogger.Error().AnErr("Error parsing package version output", err).Send()
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error parsing package version output: %v", err)
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error parsing package version output: %v", err)
}
for _, p := range pkgVersionOnSystem {
@@ -411,9 +469,9 @@ func parsePackageVersion(output string, cmdCtxLogger zerolog.Logger, command *Co
}
}
if errs == nil {
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), nil
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), nil
}
return collectOutput(&cmdOutBuf, command.Name, cmdCtxLogger, command.Output.ToLog), fmt.Errorf("error parsing package version output: %v", errs)
return collectOutput(&cmdOutBuf, command.Name, command.cmdLoggers.cmdContxt, command.cmdLoggers.global, command.Output.ToLog), fmt.Errorf("error parsing package version output: %v", errs)
}
func getPackageIndexFromCommand(command *Command, name string) int {
+7
View File
@@ -0,0 +1,7 @@
package backy
func Validate(c Command) {
if c.Cmd != "" {
}
}
+43
View File
@@ -1,6 +1,7 @@
package logging
import (
"bytes"
"fmt"
"os"
"strings"
@@ -66,6 +67,48 @@ func SetLoggingWriters(logFile string) (writers zerolog.LevelWriter) {
return
}
func SetLoggingWriterForCommand(buf *bytes.Buffer, logFile string, logToConsole bool) (writers zerolog.LevelWriter) {
console := zerolog.ConsoleWriter{}
if logToConsole {
console = zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: time.RFC1123}
console.FormatLevel = func(i interface{}) string {
return strings.ToUpper(fmt.Sprintf("| %-6s|", i))
}
console.FormatMessage = func(i any) string {
if i == nil {
return ""
}
return fmt.Sprintf("MSG: %s", i)
}
console.FormatFieldName = func(i interface{}) string {
return fmt.Sprintf("%s: ", i)
}
console.FormatFieldValue = func(i interface{}) string {
return fmt.Sprintf("%s", i)
// return strings.ToUpper(fmt.Sprintf("%s", i))
}
}
fileLogger := &lumberjack.Logger{
MaxSize: 50, // megabytes
MaxBackups: 3,
MaxAge: 28, //days
Compress: true, // disabled by default
}
fileLogger.Filename = logFile
// UNIX Time is faster and smaller than most timestamps
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
// zerolog.TimeFieldFormat = time.RFC1123
writers = zerolog.MultiLevelWriter(fileLogger)
if logToConsole {
writers = zerolog.MultiLevelWriter(console, fileLogger)
}
return
}
func IsConsoleLoggingEnabled() bool {
return os.Getenv("BACKY_CONSOLE_LOGGING") == "enabled"
}
+12
View File
@@ -0,0 +1,12 @@
commands:
copyFile:
type: file
fileOperation: copy
source: /home/andrew/Projects/backy/tests/data/fileops/source.txt
destination: /home/andrew/Projects/backy/tests/data/fileops/destination.txt
copyRemoteFile:
type: file
fileOperation: copy
sourceType: rempte
source: ssh://backy@localhost:2222/home/backy/remote_source.txt
destination: /home/andrew/Projects/backy/tests/data/fileops/remote_destination.txt
+3 -2
View File
@@ -10,7 +10,8 @@ commands:
successCmd:
name: get docker version
cmd: docker
getOutput: true
outputToLog: true
output:
file: docker_version_success.txt
toLog: true
Args:
- "-v"
+1
View File
@@ -0,0 +1 @@
This is some test data.
+1
View File
@@ -0,0 +1 @@
This is some test data.
+39
View File
@@ -0,0 +1,39 @@
SSH test container
==================
This folder contains a simple Docker-based SSH server used for integration tests.
Quick start
-----------
Start the container (builds image if needed):
```bash
./start.sh
```
Stop the container:
```bash
./stop.sh
```
Access
------
- SSH endpoint: `localhost:2222`
- Test user: `backy` with password `backy` (password auth enabled)
- Root user: `root` with password `test`
- Public key `backytest.pub` is installed for both `backy` and `root`
Running tests
-------------
1. Start the container (`./start.sh`).
2. From the repo root, run your tests (example):
```bash
GO_TEST_SSH_ADDR=localhost:2222 go test ./... -v
```
If your tests rely on an SSH private key, use `tests/docker/backytest` as the private key and restrict access appropriately.
+11 -4
View File
@@ -1,4 +1,11 @@
cd ~/Projects/backy/tests/docker
docker container rm -f ssh_server_container
docker build -t ssh_server_image .
docker run -d -p 2222:22 --name ssh_server_container ssh_server_image
#!/usr/bin/env bash
set -euo pipefail
# Build and run the test SSH container from the tests/docker directory
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
docker container rm -f ssh_server_container 2>/dev/null || true
docker build -t ssh_server_image "$SCRIPT_DIR"
docker run -d -p 2222:22 --name ssh_server_container ssh_server_image
sleep 5
ssh-keyscan -p 2222 localhost > $SCRIPT_DIR/known_hosts
+8
View File
@@ -0,0 +1,8 @@
services:
ssh_server:
build: .
image: backy_ssh_server:latest
container_name: backy_ssh_server
ports:
- "2222:22"
restart: "no"
+8
View File
@@ -0,0 +1,8 @@
# localhost:2222 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.16
[localhost]:2222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZi/dbBsgthhSj9kf4SIxTZCX+thRmvSgQ+ogC0yyHvoQrIio0Pu2MA66S779LE7Zng3CJ1lxiesffqdiTdcv8mrlnwHlzshNy4trl85NOawdNpwwlY8aiYQkxifXqAXSoHlgG+AlrbLcZ4q0YbeLnwgB3eEywN+eCHuJdetNr1Lq9NkqiGKltBIVYxHofEkUj+jBTJCyPDcp7/CKSYXhcELCaSw8vBkB7+PfsYYy0+TPXspiu/euLZfiagh6ik2fMcfc0k5P4Yn0GRt5Uk4jr85lob7ucO6Q2VPJpfavSjetcqa/oL6peEOJ4/SQzzrbxmh0I/5Tq76lTtrFT+R+tfqn0+n11FkNhS85IulmpdD2kLPGE6yH+vfjYrYD5TdH0qiOltqfXuiJCekRryoU9oWS/VSDi7aEVPryCQWf0IUqj05F/sa4RRtO0k7UOYL8NwJTS8q02sCt+/W+nX03f6XNoG7K57MZlYnREUGS6scqnvktb9bwKBunDZePMwZk=
# localhost:2222 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.16
[localhost]:2222 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEy0ReNz4EO68wUiQGHmkKOwLQhAUhY9oCyzpyROUWM0FWQId124SKPPcLD3DcYHQpofdG8h11NZ8Dyk1g/mW8M=
# localhost:2222 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.16
# localhost:2222 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.16
# localhost:2222 SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.16
[localhost]:2222 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIArPKqZ3cHWzv0o6RUko8TNpr95olBJAzHRiPIvYD7bH
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
DIR="$(cd "$(dirname "$0")" && pwd)"
echo "Starting SSH test container (building if needed)..."
docker compose -f "$DIR/compose.yml" up -d --build
ssh-keyscan -p 2222 localhost > "$DIR/known_hosts"
echo "Container started on localhost:2222"
+7
View File
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
DIR="$(cd "$(dirname "$0")" && pwd)"
echo "Stopping and removing SSH test container..."
docker compose -f "$DIR/compose.yml" down --remove-orphans
echo "Stopped."
+5
View File
@@ -0,0 +1,5 @@
{{ .greeting }}, {{ .name }}!
port: {{ .port | default "8080" }}
envHOME: {{ env "HOME" "" }}
debugYaml:
{{ toYaml . }}
+3
View File
@@ -0,0 +1,3 @@
name: Alice
greeting: Hello
port: 9090