TurboGears 2.3.0b2

This is the second upgrade in the process. If you have any questions, tag @amilworks on here. Current Version: TurboGears2==2.2.0 Upgrade Version: TurboGears2==2.3.0b2

As you are going through, check off what Step you completed to make it easier for us to help, and for you to feel accomplished.

Getting Started

Step 1. Clone the Repo

GOAL

• [x] Start by cloning the branch from this current repo

git clone -b tg-upgrade --single-branch https://github.com/UCSB-VRL/bisqueUCSB.git

cd tg-upgrade and you should see a folder structure like this

.
├── boot
├── builder
├── COPYRIGHT
├── DOCKER-BUILD_LOG.md
├── Dockerfile.caffe.xenial
├── DOCKER-RUN_LOG.md
├── entry.sh
├── LICENSE
├── Makefile
├── modules
├── README.md
├── requirements.txt
├── run-bisque.sh
├── source
├── sources.list
├── start-bisque.sh
└── virtualenv.sh

4 directories, 13 files

Step 2. Verify Everything Works

GOAL

• [x] Establish a baseline that BisQue works on your local system before changing anything.

You have two choices to verify that the repo is working.

Option 1. Pull the Latest Docker Build (Recommended)

The repo is rebuilt and pushed automatically after a successful push/merge on GitHub. Download the latest image by running the following command:

docker pull amilworks/bisque-tg2-upgrade:git

You can run this image to verify BisQue is working and able to run locally on your system by running:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste the logs below and we will take a look.

Option 2. Build the Image

Building the image from scratch is what you will end up doing throughout the upgrade process. Luckily, we automated it to where you run a Makefile.

make bisque05-caffe-xenial

If you vim Makefile, you will see it's a series of basic commands. I suggest you Edit this file and --tag the image something different and update the version you are upgrading instead of amilworks/bisque05-caffe-xenial:flour-tg2.2. You can leave it but it's kind of long. Once the build is done, verify that BisQue comes up on your local system just as above:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste both the Build and Run Logs below and we will take a look.

Step 3. Make Changes/Break Stuff

GOAL

• [ ] Upgrade TurboGears to TurboGears2==2.3.0b2, fix problems that arise, document changes.

Create a new branch Creating a new branch will enable you to make changes, nuke your own branch over and over again, and maintain a clean copy of the original branch when you need to start fresh for the millionth time.

git checkout -b [name_of_your_new_branch]

Open source/requirements.txt with your favorite editor (hopefully Vim). Change the version to the one you are assigned, and save the file.

Here is the link to the line you will update.

TurboGears2==2.2   # <--- Update Version
TurboMail==3.0.3
WebError==0.10.3
WebFlash==0.1a9
WebHelpers==1.3
WebOb==1.1.1       # <--- Potential Update Needed, Check Build Logs

Push to GitHub You made changes and you want to have version control. Push your new branch and changes to GitHub by running:

git push -u origin  [name_of_your_new_branch]

Step 4. Build and Run!

GOAL

• [ ] Build and run your changes, fix any dependency issues, document changes, rebuild if necessary (most likely unless you are super lucky), trust the process, and always ask for help if you get stuck!

Once you made your changes, make sure you cd .. out of source/ and are back to the tree view displayed above. You should see a file called Makefile. You can Edit this file as mentioned in Option 2. or simply run the following command to build the image with your updated TurboGears2.

make bisque05-caffe-xenial

While its building, make sure that there are no errors during the pip installs. Typically errors are skipped over in favor of a successful build. I know, right?

Next, once the build is finished, run the image and verify BisQue comes up on your local system:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

NOTE: There might be other packages that need updating when upgrading to different TurboGears2 versions. Sometimes there will be an error raised with the version required, or it simply will not run and you will have to do a logsdump here and we will have to sift through it together to track the failed step(s).

If everything is successful, push your branch to GitHub if you have not already, file a pull request, and pat yourself on the back. Job well done. We will review it and test it on our production-like setup and give feedback if any issues arise.

Post Upgrade

Use this issue to document any issues you faced during the upgrade process. Log dumps can be thrown here as well. Try to use the code syntax for large blocks of code so we can easily read through it. Otherwise, go crazy.

• [x] What broke, if anything? Just a simple list will suffice.
• [ ] Were there any files significantly changed or that requires significant attention?
• [ ] Can we move forward to the next upgrade?
• [ ] Any blockers now or that you foresee?

TurboGears 2.2.1

This is the second upgrade in the process. If you have any questions, tag @amilworks on here. Current Version: TurboGears2==2.2.0 Upgrade Version: TurboGears2==2.2.1

As you are going through, check off what Step you completed to make it easier for us to help, and for you to feel accomplished.

Getting Started

Step 1. Clone the Repo

GOAL

• [ ] Start by cloning the branch from this current repo

git clone -b tg-upgrade --single-branch https://github.com/UCSB-VRL/bisqueUCSB.git

cd tg-upgrade and you should see a folder structure like this

.
├── boot
├── builder
├── COPYRIGHT
├── DOCKER-BUILD_LOG.md
├── Dockerfile.caffe.xenial
├── DOCKER-RUN_LOG.md
├── entry.sh
├── LICENSE
├── Makefile
├── modules
├── README.md
├── requirements.txt
├── run-bisque.sh
├── source
├── sources.list
├── start-bisque.sh
└── virtualenv.sh

4 directories, 13 files

Step 2. Verify Everything Works

GOAL

• [ ] Establish a baseline that BisQue works on your local system before changing anything.

You have two choices to verify that the repo is working.

Option 1. Pull the Latest Docker Build (Recommended)

The repo is rebuilt and pushed automatically after a successful push/merge on GitHub. Download the latest image by running the following command:

docker pull amilworks/bisque-tg2-upgrade:git

You can run this image to verify BisQue is working and able to run locally on your system by running:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste the logs below and we will take a look.

Option 2. Build the Image

Building the image from scratch is what you will end up doing throughout the upgrade process. Luckily, we automated it to where you run a Makefile.

make bisque05-caffe-xenial

If you vim Makefile, you will see it's a series of basic commands. I suggest you Edit this file and --tag the image something different and update the version you are upgrading instead of amilworks/bisque05-caffe-xenial:flour-tg2.2. You can leave it but it's kind of long. Once the build is done, verify that BisQue comes up on your local system just as above:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste both the Build and Run Logs below and we will take a look.

Step 3. Make Changes/Break Stuff

GOAL

• [ ] Upgrade TurboGears to TurboGears2==2.2.1, fix problems that arise, document changes.

Create a new branch Creating a new branch will enable you to make changes, nuke your own branch over and over again, and maintain a clean copy of the original branch when you need to start fresh for the millionth time.

git checkout -b [name_of_your_new_branch]

Open source/requirements.txt with your favorite editor (hopefully Vim). Change the version to the one you are assigned, and save the file.

Here is the link to the line you will update.

TurboGears2==2.2   # <--- Update Version
TurboMail==3.0.3
WebError==0.10.3
WebFlash==0.1a9
WebHelpers==1.3
WebOb==1.1.1       # <--- Potential Update Needed, Check Build Logs

Push to GitHub You made changes and you want to have version control. Push your new branch and changes to GitHub by running:

git push -u origin  [name_of_your_new_branch]

Step 4. Build and Run!

GOAL

• [ ] Build and run your changes, fix any dependency issues, document changes, rebuild if necessary (most likely unless you are super lucky), trust the process, and always ask for help if you get stuck!

Once you made your changes, make sure you cd .. out of source/ and are back to the tree view displayed above. You should see a file called Makefile. You can Edit this file as mentioned in Option 2. or simply run the following command to build the image with your updated TurboGears2.

make bisque05-caffe-xenial

While its building, make sure that there are no errors during the pip installs. Typically errors are skipped over in favor of a successful build. I know, right?

Next, once the build is finished, run the image and verify BisQue comes up on your local system:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

NOTE: There might be other packages that need updating when upgrading to different TurboGears2 versions. Sometimes there will be an error raised with the version required, or it simply will not run and you will have to do a logsdump here and we will have to sift through it together to track the failed step(s).

If everything is successful, push your branch to GitHub if you have not already, file a pull request, and pat yourself on the back. Job well done. We will review it and test it on our production-like setup and give feedback if any issues arise.

Post Upgrade

Use this issue to document any issues you faced during the upgrade process. Log dumps can be thrown here as well. Try to use the code syntax for large blocks of code so we can easily read through it. Otherwise, go crazy.

• [x] What broke, if anything? Just a simple list will suffice.
• [ ] Were there any files significantly changed or that requires significant attention?
• [ ] Can we move forward to the next upgrade?
• [ ] Any blockers now or that you foresee?

TurboGears 2.2.2

This is the second upgrade in the process. If you have any questions, tag @amilworks on here. Current Version: TurboGears2==2.2.0 Upgrade Version: TurboGears2==2.2.2

As you are going through, check off what Step you completed to make it easier for us to help, and for you to feel accomplished.

Getting Started

Step 1. Clone the Repo

GOAL

• [x] Start by cloning the branch from this current repo

git clone -b tg-upgrade --single-branch https://github.com/UCSB-VRL/bisqueUCSB.git

cd tg-upgrade and you should see a folder structure like this

.
├── boot
├── builder
├── COPYRIGHT
├── DOCKER-BUILD_LOG.md
├── Dockerfile.caffe.xenial
├── DOCKER-RUN_LOG.md
├── entry.sh
├── LICENSE
├── Makefile
├── modules
├── README.md
├── requirements.txt
├── run-bisque.sh
├── source
├── sources.list
├── start-bisque.sh
└── virtualenv.sh

4 directories, 13 files

Step 2. Verify Everything Works

GOAL

• [ ] Establish a baseline that BisQue works on your local system before changing anything.

You have two choices to verify that the repo is working.

Option 1. Pull the Latest Docker Build (Recommended)

The repo is rebuilt and pushed automatically after a successful push/merge on GitHub. Download the latest image by running the following command:

docker pull amilworks/bisque-tg2-upgrade:git

You can run this image to verify BisQue is working and able to run locally on your system by running:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste the logs below and we will take a look.

Option 2. Build the Image

Building the image from scratch is what you will end up doing throughout the upgrade process. Luckily, we automated it to where you run a Makefile.

make bisque05-caffe-xenial

If you vim Makefile, you will see it's a series of basic commands. I suggest you Edit this file and --tag the image something different and update the version you are upgrading instead of amilworks/bisque05-caffe-xenial:flour-tg2.2. You can leave it but it's kind of long. Once the build is done, verify that BisQue comes up on your local system just as above:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste both the Build and Run Logs below and we will take a look.

Step 3. Make Changes/Break Stuff

GOAL

• [ ] Upgrade TurboGears to TurboGears2==2.2.2, fix problems that arise, document changes.

Create a new branch Creating a new branch will enable you to make changes, nuke your own branch over and over again, and maintain a clean copy of the original branch when you need to start fresh for the millionth time.

git checkout -b [name_of_your_new_branch]

Open source/requirements.txt with your favorite editor (hopefully Vim). Change the version to the one you are assigned, and save the file.

Here is the link to the line you will update.

TurboGears2==2.2   # <--- Update Version
TurboMail==3.0.3
WebError==0.10.3
WebFlash==0.1a9
WebHelpers==1.3
WebOb==1.1.1       # <--- Potential Update Needed, Check Build Logs

Push to GitHub You made changes and you want to have version control. Push your new branch and changes to GitHub by running:

git push -u origin  [name_of_your_new_branch]

Step 4. Build and Run!

GOAL

• [ ] Build and run your changes, fix any dependency issues, document changes, rebuild if necessary (most likely unless you are super lucky), trust the process, and always ask for help if you get stuck!

Once you made your changes, make sure you cd .. out of source/ and are back to the tree view displayed above. You should see a file called Makefile. You can Edit this file as mentioned in Option 2. or simply run the following command to build the image with your updated TurboGears2.

make bisque05-caffe-xenial

While its building, make sure that there are no errors during the pip installs. Typically errors are skipped over in favor of a successful build. I know, right?

Next, once the build is finished, run the image and verify BisQue comes up on your local system:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

NOTE: There might be other packages that need updating when upgrading to different TurboGears2 versions. Sometimes there will be an error raised with the version required, or it simply will not run and you will have to do a logsdump here and we will have to sift through it together to track the failed step(s).

If everything is successful, push your branch to GitHub if you have not already, file a pull request, and pat yourself on the back. Job well done. We will review it and test it on our production-like setup and give feedback if any issues arise.

Post Upgrade

Use this issue to document any issues you faced during the upgrade process. Log dumps can be thrown here as well. Try to use the code syntax for large blocks of code so we can easily read through it. Otherwise, go crazy.

• [x] What broke, if anything? Just a simple list will suffice.
• [ ] Were there any files significantly changed or that requires significant attention?
• [ ] Can we move forward to the next upgrade?
• [ ] Any blockers now or that you foresee?

Bumps waitress from 0.8.8 to 2.1.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Finally the /services Error is Fixed

Huge thanks to @michael-goebel for finding and fixing the /services issue that came up while upgrading from turbogears2==2.1.5 --> turbogears2==2.2. The current way to test is simply by rebuilding the BisQue image and deploying it locally to verify that the issue is resolved. A few questions are:

• @michael-goebel How did you build and test the image? Can we just build the image as usual or did you make changes to the Dockerfile and/or in other build scripts that we need to account for?
• Do we have any existing unit tests that we can use to verify the changes? Basically, if we push this to production, we don't want other things breaking that could have been caught earlier on by writing a 5-liner unit test.
• There's 55 files that are changed. Are we merging all of these files or are there specific ones we should merge?

I will review the changes and make comments if necessary. Having a second pair of eyes and another person that can test the changes on their local setup would be useful. Maybe @satish1901 or @ASMIftekhar <3

Python 3 Branch

Also, @satish1901 which branch is the one you cleaned up with the Python 3 source code? This merge needs to be changed to the branch with the Python 3 source code. We can test Mike's changes on Python 2.7 but soon thereafter this needs to be tested on Python 3.

TurboGears 2.3.0b1

This is the second upgrade in the process. If you have any questions, tag @amilworks on here. Current Version: TurboGears2==2.2.0 Upgrade Version: TurboGears2==2.3.0b1

As you are going through, check off what Step you completed to make it easier for us to help, and for you to feel accomplished.

Getting Started

Step 1. Clone the Repo

GOAL

• [x] Start by cloning the branch from this current repo

git clone -b tg-upgrade --single-branch https://github.com/UCSB-VRL/bisqueUCSB.git

cd tg-upgrade and you should see a folder structure like this

.
├── boot
├── builder
├── COPYRIGHT
├── DOCKER-BUILD_LOG.md
├── Dockerfile.caffe.xenial
├── DOCKER-RUN_LOG.md
├── entry.sh
├── LICENSE
├── Makefile
├── modules
├── README.md
├── requirements.txt
├── run-bisque.sh
├── source
├── sources.list
├── start-bisque.sh
└── virtualenv.sh

4 directories, 13 files

Step 2. Verify Everything Works

GOAL

• [x] Establish a baseline that BisQue works on your local system before changing anything.

You have two choices to verify that the repo is working.

Option 1. Pull the Latest Docker Build (Recommended)

The repo is rebuilt and pushed automatically after a successful push/merge on GitHub. Download the latest image by running the following command:

docker pull amilworks/bisque-tg2-upgrade:git

You can run this image to verify BisQue is working and able to run locally on your system by running:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste the logs below and we will take a look.

Option 2. Build the Image

Building the image from scratch is what you will end up doing throughout the upgrade process. Luckily, we automated it to where you run a Makefile.

make bisque05-caffe-xenial

If you vim Makefile, you will see it's a series of basic commands. I suggest you Edit this file and --tag the image something different and update the version you are upgrading instead of amilworks/bisque05-caffe-xenial:flour-tg2.2. You can leave it but it's kind of long. Once the build is done, verify that BisQue comes up on your local system just as above:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste both the Build and Run Logs below and we will take a look.

Step 3. Make Changes/Break Stuff

GOAL

• [ ] Upgrade TurboGears to `TurboGears2==2.3.0b1', fix problems that arise, document changes.

Create a new branch Creating a new branch will enable you to make changes, nuke your own branch over and over again, and maintain a clean copy of the original branch when you need to start fresh for the millionth time.

git checkout -b [name_of_your_new_branch]

Open source/requirements.txt with your favorite editor (hopefully Vim). Change the version to the one you are assigned, and save the file.

Here is the link to the line you will update.

TurboGears2==2.2   # <--- Update Version
TurboMail==3.0.3
WebError==0.10.3
WebFlash==0.1a9
WebHelpers==1.3
WebOb==1.1.1       # <--- Potential Update Needed, Check Build Logs

Push to GitHub You made changes and you want to have version control. Push your new branch and changes to GitHub by running:

git push -u origin  [name_of_your_new_branch]

Step 4. Build and Run!

GOAL

• [ ] Build and run your changes, fix any dependency issues, document changes, rebuild if necessary (most likely unless you are super lucky), trust the process, and always ask for help if you get stuck!

Once you made your changes, make sure you cd .. out of source/ and are back to the tree view displayed above. You should see a file called Makefile. You can Edit this file as mentioned in Option 2. or simply run the following command to build the image with your updated TurboGears2.

make bisque05-caffe-xenial

While its building, make sure that there are no errors during the pip installs. Typically errors are skipped over in favor of a successful build. I know, right?

Next, once the build is finished, run the image and verify BisQue comes up on your local system:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

NOTE: There might be other packages that need updating when upgrading to different TurboGears2 versions. Sometimes there will be an error raised with the version required, or it simply will not run and you will have to do a logsdump here and we will have to sift through it together to track the failed step(s).

If everything is successful, push your branch to GitHub if you have not already, file a pull request, and pat yourself on the back. Job well done. We will review it and test it on our production-like setup and give feedback if any issues arise.

Post Upgrade

Use this issue to document any issues you faced during the upgrade process. Log dumps can be thrown here as well. Try to use the code syntax for large blocks of code so we can easily read through it. Otherwise, go crazy.

• [x] What broke, if anything? Just a simple list will suffice.
• [ ] Were there any files significantly changed or that requires significant attention?
• [ ] Can we move forward to the next upgrade?
• [ ] Any blockers now or that you foresee?

TurboGears 2.2.0

As a first step in the upgrade process, we will shoot for incremental upgrades of TG2 and document any issues faced.

Current Version: TurboGears2==2.1.5 Upgrade Version: TurboGears2==2.2.0

Post Upgrade

Use this issue to document any issues you faced during the upgrade process. Log dumps can be thrown here as well. Try to use the code syntax for large blocks of code so we can easily read through it. Otherwise, go crazy.

• What broke, if anything? Just a simple list will suffice.
• Were there any files significantly changed or that requires significant attention?
• Can we move forward to the next upgrade?
• Any blockers now or that you foresee?

• I have uploaded a file,
• made it publicly accessible
• Opened the image and tried to rename it using the button on the top right corner
• Changes are not being reflected

We need to find the root cause of this issue. Not sure what's happening in the backend.

Thanks, Chandrakanth.

Description: Unable to delete the files that belong to a datasets

Steps to Recreate:

• Upload two images and add to a dataset.

• Navigate to that dataset

• Click on an image and click delete button (on top left corner)

• You should see a pop up saying Resource deleted successfully! You will be redirected to your BISQUE homepage.

• But the file is not getting deleted. It is still sitting inside the dataset

Note: There is no issue in deleting the files that do NOT belong to a dataset.

Thanks.

Jupyter notebooks to guide users on using bqapi for file upload and file download.

Potential TODOs:

1. Functionality to upload folder/dataset
2. Functionality to download folder/dataset
3. Clear documentation on several response codes

It would be really helpful for the first time users of Bisque to have a default inputs show up automatically, when the user clicks on buttons like Select an Image or Select file (while working on Bisque modules).

Current state: User have to manually investigate About/Help tab to figure out the appropriate input format (.png or .tif or .xyz). It would be great to have appropriate inputs, to be selected, show up automatically. So users cannot get an actual sense of each module as they do not have access to first-good sample input.

Posting this here. But this could be a change to module generator repository.