Commit graph

771 commits

Author SHA1 Message Date
zhengtianbao
e35a87e3eb
Update registry template (#8198)
* Add registry replica setting

* Add registry liveness and readiness probe

* Set the security context for registry

* Add registry pvc access mode option

* registry add replica requirement check

* docs: add registry replicas setting note

* Update docs/kubernetes-apps/registry.md

Co-authored-by: Cristian Calin <6627509+cristicalin@users.noreply.github.com>

Co-authored-by: Cristian Calin <6627509+cristicalin@users.noreply.github.com>
2021-11-22 02:45:09 -08:00
Mathieu Parent
0263c649f4
Allow to scrape etcd metrics using a service (#8203)
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
2021-11-17 23:53:01 -08:00
IKRozhkov
2c87170ccf
Allow setting 'auto-assign' property to 'false' for default IP pool (Metallb addon) (#8193)
* add metallb auto-assign property for main IP range & update addons.yml for sample inventory

* add new line at the end of file roles\kubernetes-apps\metallb\defaults\main.yml

* set default value for matallb_auto_assign = true
2021-11-16 05:06:27 -08:00
zhengtianbao
02322c46de
Remove helm duplicate check (#8196) 2021-11-15 12:50:48 -08:00
Samuel Liu
027cbefb87
change krew uri to krew_download_url (#8190) 2021-11-14 12:08:47 -08:00
zhengtianbao
5f1456337b
Fix krew auto completion command not found at lower version (#8185) 2021-11-12 17:04:46 -08:00
Ajarmar
b5a5478a8a
Added tolerations for cinder-csi-nodeplugin DaemonSet (#8137) 2021-11-11 11:48:07 -08:00
Cristian Calin
039205560a
nodelocaldns: allow a secondary pod for nodelocaldns for local-HA (#8100)
* nodelocaldns: allow a secondary pod for nodelocaldns for local-HA

* CI: add job to test nodelocaldns secondary
2021-11-09 09:57:47 -08:00
zhengtianbao
46c536d261
Add krew auto completion (#8171) 2021-11-09 02:43:39 -08:00
zhengtianbao
65540c5771
krew: update to v0.4.2 (#8168)
krew release urls changed since v0.4.2, clearly OS type and arch inside the filename.

from:
  https://github.com/kubernetes-sigs/krew/releases/download/v0.4.1/krew.tar.gz
to:
  https://github.com/kubernetes-sigs/krew/releases/download/v0.4.2/krew-linux_amd64.tar.gz

define `host_os` like `host_architecture` determine which OS is krew
installed at.
2021-11-08 02:54:59 -08:00
Álvaro Torres Cogollo
8922c45556
Added ArgoCD kubernetes-app (#7895)
* Added ArgoCD kubernetes-app

* Update argocd_version to latest
2021-11-07 02:22:51 -08:00
Antoine Gatineau
b7eb1cf936
cert-manager: add trusted internal ca when configured (#8135)
* cert-manager: add trusted internal ca when configured

* wrong check for inventory variable

* Update documentation
2021-11-05 09:43:52 -07:00
zhengtianbao
9d4cdb7b02
Ensure addon-resizer 1.8.11 only effective at arch amd64. (#8144)
* Ensure addon-resizer 1.8.11 only effective at arch amd64.

k8s.gcr.io/addon-resizer:1.8.11 returns the amd64 image which is not executable at arm64.

Disable addon-resizer when the platform is not amd64.

When metrics-server upgrade and use addon-resizer:2.3, then revert this
commit and `image_arch` will determine the `addon_resizer_image_tag`.

* Add metrics_server_resizer architectures check
2021-11-01 08:21:19 -07:00
Sergey
0b441ade2c
nginx ingress controller should watch kind:ingress without class (#8128) 2021-10-28 11:48:59 -07:00
Krystian Młynek
6f6fad5a16
Calico: add missing verbs in ClusterRole (#8136) 2021-10-28 11:11:01 -07:00
irizzant
649f962ac6
Metrics-server Deployment has incongruencies in resources requests/limits (#8088)
* fix(metrics-server): update defaults

* fix(metrics-server): typo error
2021-10-27 15:15:11 -07:00
Sébastien Masset
7c3369e1b9
Fixed default DNS min replica for single node clusters (#8112) 2021-10-26 16:03:46 -07:00
Florian Ruynat
331647f4ab
Remove deprecated Ambassador ingress code (#8086) 2021-10-26 15:19:09 -07:00
Damian Szeluga
4db057e9c2
Allow changing metallb default pool name (#8111) 2021-10-22 09:38:39 -07:00
Cristian Calin
6a5b87dda4
netchecker: update images to 1.2.2 from Mirantis (#8074)
* netchecker: update images to 1.2.2 from Mirantis which is slightly less ancinet than the l23networks images

* Netchecker: use local etcd instead of kubernetes v1beta1 crds which are no longer suported by kube 1.22+
2021-10-19 10:17:04 -07:00
Olivier Levitt
7019c2685d
Increase cpu limit to prevent throttling (#8076) 2021-10-14 11:03:36 -07:00
Cristian Calin
cee481f63d
cert-manager: upgrade to 1.5.4 (#8069)
* cert-manager: update to 1.5.4

* cert-manager: remove outdated guidelines on creating an initial ClusterIssuer
2021-10-12 09:17:47 -07:00
rtsp
6c004efd5f
cert_manager: Remove deprecated ClusterIssuer and its Secret (#8064) 2021-10-11 09:40:40 -07:00
Maciej Wereski
ce25e4aa21
MetalLB: update to v0.10.3 (#8071)
Signed-off-by: Maciej Wereski <m.wereski@partner.samsung.com>
2021-10-11 08:54:40 -07:00
Rene Luria
ef4044b62f
csi_driver / cinder: implement rescan-on-resize variable via (#8057)
cinder_csi_rescan_on_resize
2021-10-11 02:14:40 -07:00
Kenichi Omichi
8d3961edbe
Add metrics_server_resizer option (#8018)
The addon-resizer container can reduce resource limits of cpu and
memory of metrics-server container in the pod, and that caused
OOMKilled.
In addition, the original metrics-server manifest doesn't contain
the addon-resizer container as [1].
So this adds metrics_server_resizer option to control the addon-resizer
container deployment and the default value is false to make it stable
for most environments.

[1]: 527679e5e8/manifests/base/deployment.yaml
2021-09-28 00:02:42 -07:00
Kenichi Omichi
d27cf375af
Remove allowPrivilegeEscalation from metrics-server (#8014)
"allowPrivilegeEscalation: false" blocks deploying metrics-server
on CentOS7. In addition, the original metrics-server manifest doesn't
contain it as [1]. This removes it.

[1]: 527679e5e8/manifests/base/deployment.yaml
2021-09-27 08:43:36 -07:00
Hugo Blom
ac846667b7
Check if openstack application credentials are empty since they always exists (#8021) 2021-09-27 01:14:22 -07:00
Florian Ruynat
c8d7f000c9
Remove k8s hooks for versions prior to 1.20 (#7998) 2021-09-22 10:32:01 -07:00
jhchong92
83f64a7ff9
Bugfix/cinder csi cloud config template (#7955)
* Fix invalid condition for username and password inclusion

* Use length filter to test variable conditions
2021-09-09 10:04:11 -07:00
jhchong92
bd8b8916a8
Remove invalid spec - deployment.spec.serviceName (#7949) 2021-09-08 13:05:56 -07:00
jhchong92
57063b6828
Replace incorrect {% end %} tags with {% endif %} in csi_crd templates (#7947) 2021-09-08 12:59:57 -07:00
Cristian Calin
43e7e2d663
nginx-ingress: bump to 1.0.0 to support kube 1.22 (#7942) 2021-09-06 04:50:36 -07:00
Cristian Calin
db470f8529
Update CSI snaphotter and make it independent (#7943)
* CSI: update CSI snapshot CRDs

* CSI: update snapshot controller tag version with kubernetes specific versions

* CSI: allow enabling csi_snapshot_controller independent of Cinder CSI

* CSI: Align csi-snapshot-controller with upstream and use a Deployment instead of a StatefulSet
2021-09-06 04:24:29 -07:00
Maciej Wereski
e1967b0700
MetalLB: keep nodeSelector in one place (#7931)
Signed-off-by: Maciej Wereski <m.wereski@partner.samsung.com>
2021-09-01 09:05:00 -07:00
Olivier Lemasle
507091ec8b
Replace cluster_name by dns_domain (#7923)
`cluster_name` defaults to `dns_domain` value (see [here][1] and [here][2])
but they could have different values.

`dns_domain` should be used here instead of `cluster_name` because the DNS
resolution is configured to use `dns_domain`.

[1]: 0ef7af76bc/roles/kubespray-defaults/defaults/main.yaml (L104)
[2]: 1afdb05ea9/inventory/sample/group_vars/k8s_cluster/k8s-cluster.yml (L196)
2021-09-01 08:18:59 -07:00
Maciej Wereski
48ceca4919
MetalLB: update to v0.10.2 (#7925)
Signed-off-by: Maciej Wereski <m.wereski@partner.samsung.com>
2021-09-01 03:00:59 -07:00
Sergey
89993e4833
fix error metrics server capabilities name (#7905) 2021-08-25 12:06:15 -07:00
rtsp
4c9d7dedb3
addons/cert_manager: retries until webhook pods has been created (#7850)
Fix task 'Cert Manager | Wait for Webhook pods become ready' failed due to webhook pods don't exist yet by using `retries..until` trick like kubernetes-sigs/kubespray#7842

This fix should be removed in the future if the kubernetes/kubernetes#83242 is resolved.

Signed-off-by: rtsp <git@rtsp.us>
2021-08-25 07:16:41 -07:00
Léopold Jacquot
c06896a352
Update metrics-server to 0.5.0 (#7864) 2021-08-12 08:19:48 -07:00
Eugene Artemenko
e2b67b5700
Add suport of Vsphere CSI driver 2.2.X versions (#7848) 2021-08-09 08:19:38 -07:00
rtsp
82a9064d8d
addons/cert_manager: fix kubernetes-sigs#7085 by adding retries..until (#7842)
Fix task 'Cert Manager | Apply ClusterIssuer manifest' failed due to service/endpoints updating delayed even though the wekhook pod status is ready.

Signed-off-by: rtsp <git@rtsp.us>
2021-08-09 08:19:31 -07:00
Vitaliy D
5db86f4c2b
Update vSphere CPI (#7838)
Changes:
  * ClusterRole updated according to the latest manifests from
    https://github.com/kubernetes/cloud-provider-vsphere
  * vSphere CPI/CSI default versions bumped and
    tested successfully on K8S 1.21.1
  * vSphere documentation updated

Signed-off-by: Vitaliy D <vi7alya@gmail.com>
2021-07-29 18:17:37 -07:00
Fredrik Liv
802fb8b591
Add application credentials support for cinder (#7799)
* csi-driver: Added possibility to use application credentials for cinder

* external-cloud-controller: Added env vars for openstack application credentials
2021-07-15 00:56:48 -07:00
Cristian Calin
7516fe142f
Move to Ansible 3.4.0 (#7672)
* Ansible: move to Ansible 3.4.0 which uses ansible-base 2.10.10

* Docs: add a note about ansible upgrade post 2.9.x

* CI: ensure ansible is removed before ansible 3.x is installed to avoid pip failures

* Ansible: use newer ansible-lint

* Fix ansible-lint 5.0.11 found issues

* syntax issues
* risky-file-permissions
* var-naming
* role-name
* molecule tests

* Mitogen: use 0.3.0rc1 which adds support for ansible 2.10+

* Pin ansible-base to 2.10.11 to get package fix on RHEL8
2021-07-12 00:00:47 -07:00
Cristian Calin
bd6d810d0a
nodelocaldns: allow binding metrics address to host IP (#7748) 2021-06-29 05:28:41 -07:00
Id2ndR
a31baf3c16
Fix deployment without openstack cacert (#7723)
* fix group name

* fix external-openstack-cloud-config secret

* don't add ca.cert in the secret if not defined
2021-06-21 05:38:50 -07:00
Cristian Calin
282a27a07c
gVisor: initial support for gVisor container runtime (#7661)
* Docker/Containerd: move downloads urls to containerd-common

* gVisor: initial support for gVisor container runtime
2021-06-21 05:18:51 -07:00
Samuel Liu
7f7e83a4d9
fix local-path-provisioner helper image repo (#7703) 2021-06-16 08:06:00 -07:00
Cristian Calin
6a2ea94b39
Docs improvements (#7660)
* Docs: update sidebar

* Docs: move registry documentation into docs/

* Docs: move rbd_provisioner documentation into docs/

* Docs: move cephfs_provisioner into docs/

* Docs: move local_volume_provisioner documentation into docs/

* Docs: move ambassador.md to docs/ingress_controller/

* Docs: move metallb.md to docs/ingress_controller/

* Docs: move ingress_nginx documentation into docs/

* Docs: move alb_ingress_controller documentation into docs/

* Docs: merge ambassador documentation into docs/ingress_controller/

* Docs: move cert_manager documentation into docs/

* Docs: move bootstrap-os documentation into docs/

* Docs: update file locations in sidebar
2021-06-01 07:30:27 -07:00
Cedric Hnyda
4674b03661
Add cinder_csi_ignore_volume_az (#7624)
Signed-off-by: Cedric Hnyda <cedric.hnyda@itera.io>
2021-06-01 07:10:27 -07:00
kongxs
922de32290
spelling mistakes (#7664)
Signed-off-by: kjinan <2008kongxiangsheng@163.com>
2021-05-31 05:46:26 -07:00
Fredrik Liv
3ca205446e
Added possibility to specify vSphere credentials via env variables (#7646)
* Added possibility to specify vSphere credentials via env variables

* Removed excessive spacing
2021-05-27 12:02:30 -07:00
Cristian Calin
c5ccedb694
store openstack external cloud controller ca.cert in a k8s secret instead of the host filesystem (#7603) 2021-05-26 00:35:21 -07:00
Cristian Calin
858b29f425
Calico: add support for v3.19.1 (#7630)
* Calico: add v3.19.1 hashes

* enable liveness probe for calico-kube-controllers

3.19.1

* Calico: drop support for v3.16.x

* Calico: promote v3.18.3 as default
2021-05-25 13:40:50 -07:00
efrikin
7db76f8809
Add nodeSelctor for other services and node labels before CNI setup (#7613) 2021-05-25 13:40:43 -07:00
Cristian Calin
e9870b8d25
add support for using ansible 2.10.x for deploying kubespray (#7600)
* add support for using ansible 2.10.x for deploying kubespray

* move dns-autoscaler-clusterrole{binding}.yml to files/ folder

* note that ansible 2.10 is now experimentally supported

* coredns: move files to templates like before #4341
2021-05-18 05:39:31 -07:00
Cristian Calin
14cf3e138b
Support Calico advertisement of MetalLB LoadBalancer IPs (#7593)
* add initial MetalLB docs

* metallb allow disabling the deployment of the metallb speaker

* calico>=3.18 allow using calico to advertise service loadbalancer IPs

* Document the use of MetalLB and Calico

* clean MetalLB docs
2021-05-12 05:22:17 -07:00
Cedric Hnyda
1dd38721b3
Add external_openstack_enable_ingress_hostname option for openstack (#7572)
Signed-off-by: Cedric Hnyda <cedric.hnyda@itera.io>
2021-05-04 00:33:11 -07:00
Eugene Artemenko
6a001e4971
Add suport of Vsphere CSI driver 2.X versions (#7480) 2021-05-04 00:05:11 -07:00
Samuel Liu
96e6a6ac3f
Add krew support (#7464)
* Add krew support

* Add reset for krew

* Update install krew(local)

* ansible lint

* yamllint

* fix krew default vars

* fix kubectl_localhost mode

* replace include

* fix e206
2021-05-03 07:16:03 -07:00
bac-w
2556eb2733
Upgrade cilium role (#7521)
* Upgrade cilium roles

* Del old test result

* Add hubble ui examples

* Refactor hubble metrics

* Markdown fix pipeline errors

* yamllint check and fix

* refactor install from https://github.com/kubernetes-sigs/kubespray/pull/7520

* Docs syntax change (fix)

* Cilium set default 1.8.9

* Update cilium version in Readme
2021-04-30 08:09:59 -07:00
holmesb
3f4eb9be08
Fixes issue #7573 - Made Calico permissions compatible with v3.18.x (see https://github.com/projectcalico/calico/issues/4557). Specifically, granted watch to custom resources blockaffinities, ipamblocks & ipamhandles (#7575) 2021-04-30 07:25:59 -07:00
Cristian Calin
360aff4a57
Rename ansible groups to use _ instead of - (#7552)
* rename ansible groups to use _ instead of -

k8s-cluster -> k8s_cluster
k8s-node -> k8s_node
calico-rr -> calico_rr
no-floating -> no_floating

Note: kube-node,k8s-cluster groups in upgrade CI
      need clean-up after v2.16 is tagged

* ensure old groups are mapped to the new ones
2021-04-29 05:20:50 -07:00
Florian Ruynat
c16efc9ab8
Fix Opensuse not working with ansible_distribution (#7551) 2021-04-26 08:37:02 -07:00
Samuel Liu
8c7b90ebbf
add ingress controller class (#7522) 2021-04-22 00:22:38 -07:00
Ian Martin
38d9d2ea0e
Ambassador can watch multiple namespaces (#7516)
* Ambassador can watch multiple namespaces

* update variable name per PR review
2021-04-22 00:22:31 -07:00
Samuel Liu
90c643f3ab
format ansible output (#7482) 2021-04-11 00:37:59 -07:00
orange-llajeanne
7e75d48cc4
local provisioner 'useNodeNameOnly' option can be configured (#7421) 2021-04-01 16:54:11 -07:00
rptaylor
7dec8e5caa
specify runAsGroup, allow safe sysctls by default (#7399) 2021-03-25 08:03:30 -07:00
Kenichi Omichi
486b223e01
Replace kube-master with kube_control_plane (#7256)
This replaces kube-master with kube_control_plane because of [1]:

  The Kubernetes project is moving away from wording that is
  considered offensive. A new working group WG Naming was created
  to track this work, and the word "master" was declared as offensive.
  A proposal was formalized for replacing the word "master" with
  "control plane". This means it should be removed from source code,
  documentation, and user-facing configuration from Kubernetes and
  its sub-projects.

NOTE: The reason why this changes it to kube_control_plane not
      kube-control-plane is for valid group names on ansible.

[1]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint/README.md#motivation
2021-03-23 17:26:05 -07:00
Maciej Wereski
4f89bfac48
MetalLB: bump to v0.9.6 (#7397)
Signed-off-by: Maciej Wereski <m.wereski@partner.samsung.com>
2021-03-23 13:42:06 -07:00
p53
5fee96b404
Fix cinder cert permissions (#7384)
* Fix permissions of cinder cert

* Change runuser for external_cloud_controller to kube user with id 999, part of 999 - kube-cert group
2021-03-23 11:03:37 -07:00
François Hernandez
c7db72e1da
Add nodeselector and tolerations for metallb (#7334)
* add nodeselector and tolerations for metallb

* remove unnecessary commented lines in metallb template

* set default speaker toleration to match original manifest
2021-03-08 07:57:42 -08:00
Etienne Champetier
8800b5c01d Remove rotate_tokens logic
kubeadm never rotates sa.key/sa.pub, so there is no need to delete tokens/restart pods

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
2021-03-04 23:42:22 -08:00
Sergey
2c9fc18903
template crun manifest (#7305)
add missing else to if inline
2021-03-02 01:57:19 -08:00
Florian Ruynat
100d9333ca
Add configmaps to local-path-provisioner CR (#7323) 2021-02-25 16:22:17 -08:00
Sergey Mikhaltsov
a4cc416511
use external_openstack_lbaas_use_octavia for template openstack-cloud… (#7298)
* use external_openstack_lbaas_use_octavia for template openstack-cloud-config

* Delete external_openstack_lbaas_use_octavia from default values. Added description and default values of variables to docs

* markdown fix

* make this simple

* set external_openstack_lbaas_use_octavia in default values

* duplicated variable in doc
2021-02-25 11:25:25 -08:00
Etienne Champetier
3749729d5a
Remove calico-upgrade leftovers (#7282)
This is dead code since 28073c76ac

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
2021-02-16 11:24:58 -08:00
Hugo Blom
17143dbc51
write openstack controller manifests with correct perms (#7284) 2021-02-15 00:53:05 -08:00
Etienne Champetier
de1d9df787
Only use stat get_checksum: yes when needed (#7270)
By default Ansible stat module compute checksum, list extended attributes and find mime type
To find all stat invocations that really use one of those:
git grep -F stat. | grep -vE 'stat.(islnk|exists|lnk_source|writeable)'

Signed-off-by: Etienne Champetier <e.champetier@ateme.com>
2021-02-10 05:36:59 -08:00
Sergey
a21ee33180
fix typo error in role ingress-nginx (#7272) 2021-02-09 07:53:13 -08:00
David Louks
0cc1726781
Remove deletion of coredns deployment. (#7211)
* Add unique annotation on coredns deployment and only remove existing deployment if annotation is missing.

* Ignore errors when gathering coredns deployment details to handle case where it doesn't exist yet

* Remove run_once, deletegate_to and add to when statement
2021-02-09 06:02:40 -08:00
Sebastian Schmid
1f84d6344b
local-path-provisioner change default version to v0.0.19 and update config template (#7238)
* update local-path-storage config template to version v0.0.19

* changes local_path_provisioner image tag to v0.0.19

* removes copy paste example from rancher local-path-provisioner repo
2021-02-03 06:50:28 -08:00
Maciej
b42bf39fb7
MetalLB: bump to v0.9.5 (#7241)
Signed-off-by: Maciej Wereski <m.wereski@partner.samsung.com>
2021-02-03 01:02:28 -08:00
Florian Ruynat
9007d6621a
Update nginx, minor weave and misc CI tools (vagrant/terraform) (#7215) 2021-01-26 08:22:34 -08:00
Florian Ruynat
ef939dee74
Add missing 'ingress-controller' tag to alb (#7204) 2021-01-22 19:11:39 -08:00
Florian Ruynat
222a77dfe7
Change node-role.kubernetes.io from master to control-plane (#7183) 2021-01-21 08:13:03 -08:00
Florian Ruynat
9a75501152 Promote node.k8s.io API groups from v1beta1 to v1 2021-01-19 08:57:45 -08:00
Florian Ruynat
f6fbbc17a4
Cleanup old checks for k8s 1.18 (#7192) 2021-01-19 08:43:45 -08:00
Florian Ruynat
b493c81ce8
Update metrics-server to 0.4.1 (#7188) 2021-01-19 05:45:44 -08:00
Sergey
02213d6e07
change nodeSelector label from deprecated beta.kubernetes.io/os and arch to kubernetes.io prefix (#7138) 2021-01-13 13:55:02 -08:00
Kenichi Omichi
398a995798
Fix markdownlint failures under ./roles/ (#7089)
This fixes markdownlint failures under roles/
2020-12-30 05:07:49 -08:00
François Travais
98b43bb24f
Removes apps tags from apps meta dependencies (#7041)
Signed-off-by: François Travais <francois@travais.fr>
2020-12-19 08:14:25 -08:00
Cedric Hnyda
db17ba54b4
Add cluster-name to external-openstack-cloud-controller-manager (#7055)
If cluster-name is not set, the default value "kubernetes" is used.
The loadbalancees created by Kubernetes follow the format:
  kube_service_clusterName_serviceNamespace_serviceName
If 2 clusters create a loadbalancer for the same service in the same
namespace, they will share the same non-working loadbalancer.

Signed-off-by: Cedric Hnyda <cedric.hnyda@itera.io>
2020-12-17 08:23:09 -08:00
Etienne Champetier
68b96bdf1a
Helm v3 only (#6846)
* Fix etcd download dest

Signed-off-by: Etienne Champetier <champetier.etienne@gmail.com>

* Only support Helm v3, cleanup install

Signed-off-by: Etienne Champetier <champetier.etienne@gmail.com>
2020-12-02 00:20:50 -08:00
Victor Morales
4f7a760a94
Add crun support (#6864)
Signed-off-by: Victor Morales <v.morales@samsung.com>
2020-12-01 11:00:50 -08:00
Barry Melbourne
f6a5948f58
Upgrade Jetstack Cert-Manager v1.0.4 (#6937) 2020-11-30 06:52:50 -08:00
Florian Ruynat
f6eed8091e
Remove contiv related files (#6964) 2020-11-30 06:48:50 -08:00
Dmitry Chusovitin
c09aabab0c
Remove executable bit from yaml and j2 files (#6894) 2020-11-29 20:18:48 -08:00