vBulletin 5 Upgrade

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • theSven
    Master of None
    • Jan 2014
    • 855

    vBulletin 5 Upgrade

    The project to upgrade vBulletin to version 5 has started, and this thread is to track the upgrade from my test environment into production.

    I built a local Ubuntu 22.04 virtual machine, downloaded the DB backup, the forum files, and I have bought a new license for vBulletin 5. I got everything in place, but was not able to get the forum running due to PHP not working with the old code. I tried upgrading to MariaDB and PHP 8.1, but the latest PHP is not compatible with the forum code. However getting the most recent version of vBulletin is supported on the latest PHP. It was a fun Sunday evening toying with Ubuntu, I haven't used that OS in 10 years or longer. Ubuntu is the choice for the forum software due to having the latest supported versions of PHP which the forum software requires.

    At the moment I'm running through the update, it has been over 10 minutes, but there are a lot of database schema updates happening. After the first upgrade I need to run through the upgrade again and update my build steps to make sure I have that tracked. I also need to get in touch with Multacom about how to upload an ISO to update the OS on the current server, or lease a new server to use and then keep the older server as a backup... Decisions :-)
    Painter in training
  • Kevin P
    Member
    • Aug 2000
    • 10809

    #2
    I'd look into leasing a new server, or VPS, so you can transition over with minimal downtime and disruption to the existing site. As I recall the existing hardware doesn't support virtualization. With VMs you can install your own OS and not be limited to whatever the hosting provider supports. Plus you could set up a separate test and prod instance, maybe separate things like the database and email for security, etc. depending on how ambitious you are. Having the ability to take snapshots in a VM manager, or on a filesystem like ZFS, would make recovery easier and backups more efficient.

    Also the existing hardware is close to 10 years old, or older. It was provisioned in 2013. Maybe time to freshen things up some.

    I wanted to go Ubuntu with this server, but the hosting provider or DirectAdmin didn't support it so that's why the server has Debian Lenny, which is way past EOL now.

    That's just my 0.02.

    Comment

    • theSven
      Master of None
      • Jan 2014
      • 855

      #3
      Originally posted by Kevin P
      I'd look into leasing a new server, or VPS, so you can transition over with minimal downtime and disruption to the existing site. As I recall the existing hardware doesn't support virtualization. With VMs you can install your own OS and not be limited to whatever the hosting provider supports. Plus you could set up a separate test and prod instance, maybe separate things like the database and email for security, etc. depending on how ambitious you are. Having the ability to take snapshots in a VM manager, or on a filesystem like ZFS, would make recovery easier and backups more efficient.

      Also the existing hardware is close to 10 years old, or older. It was provisioned in 2013. Maybe time to freshen things up some.

      I wanted to go Ubuntu with this server, but the hosting provider or DirectAdmin didn't support it so that's why the server has Debian Lenny, which is way past EOL now.

      That's just my 0.02.
      Thank you for the 0.02 cents!!! I agree that a new server is looking very favorable. My other thought was maybe using AWS with external database. Then I can deploy new instances of the forum and not maintain servers anymore. I can leverage Ansible to automate the builds, create the ec2 instance, install the forum software, etc. Containers would be where I would uiitmately like to get to as that is where all the software I use is headed. Do you you have any thoughts on keeping the DNS server that is there too? I'm wondering if using go daddy for the DNS would be the way to go moving foward...
      Painter in training

      Comment

      • theSven
        Master of None
        • Jan 2014
        • 855

        #4
        I successfully updated the forum yesterday. Some findings:
        - The template styling that is on the forum vb4 no longer works and some theme/template work may be required
        - All vb4 plugins will no longer work, no more likes, thanks, laugh with the old plugin
        - There are new spam controls to integrate with 3rd party providers akismet and Stop Forum Spam
        - Still need to implement and test TLS configuration
        - Email testing still required
        - DNS server needs to be updated if kept and implement DNSSEC
        Painter in training

        Comment

        • Kevin P
          Member
          • Aug 2000
          • 10809

          #5
          The current server has its own BIND9 installed for DNS (maintainable via DirectAdmin when that worked), but you can set up the DNS however you like in your new setup.

          There may be another like/thanks plugin that works with VB5. The one we have now is just a trial version of something that i hacked to change something else to "laugh".

          Spam control is super important. The setup I have now blocks the majority of spam registration attempts. Very few get through and contaminate the forum. By the way, when the forum blocks a spammer, the way I coded it, it still creates the user but it puts them into the "spammer dungeon" usergroup. This way that username can't be reused to register again. I also have it coded so that any future attempts to register from the same email address or IP address of a dungeoned user will also be rejected. I generally go in and prune the dungeoned users after a year, and suspicious and unverified (never activated their accounts by clicking the email link) after 90 days.

          Comment

          • theSven
            Master of None
            • Jan 2014
            • 855

            #6
            Originally posted by Kevin P
            The current server has its own BIND9 installed for DNS (maintainable via DirectAdmin when that worked), but you can set up the DNS however you like in your new setup.

            There may be another like/thanks plugin that works with VB5. The one we have now is just a trial version of something that i hacked to change something else to "laugh".

            Spam control is super important. The setup I have now blocks the majority of spam registration attempts. Very few get through and contaminate the forum. By the way, when the forum blocks a spammer, the way I coded it, it still creates the user but it puts them into the "spammer dungeon" usergroup. This way that username can't be reused to register again. I also have it coded so that any future attempts to register from the same email address or IP address of a dungeoned user will also be rejected. I generally go in and prune the dungeoned users after a year, and suspicious and unverified (never activated their accounts by clicking the email link) after 90 days.
            Thanks Kevin :-)

            DNS is migrated to GoDaddy now, they had the cheapest solution with the features that I was looking for. The price for email hosting was steep for what is needed. I'm looking at Rackspace which has accounts for a very low monthly fee and provides spam controls. Outsourcing those two only leaves the DB and web server needed to run this site.

            I now have a test vb5 with a backup of the db from earlier in June updated. TLS is working,I have also added Sphinx Search, and exported all the attachments out of the DB into filesystem which reduces the DB size for the backups. I need to test out the new spam controls next that VB5 offers and update the backups scripts if needed that you have working. Another thing I need to do is test moving backups and rotating them on Amazon S3. I think that is where I will put them since I don't have space or a computer at home to use at this time.

            My guestimate right now is to get the server migrated in the middle of July time frame. I'm still between a new server hosted by the current host provider or deploying in AWS. AWS is a bit more attractive that way I can migrate to new OS's with patches and not have to babysit the server. I can leverage Ansible to automate the whole process. I need to do a little more research and calculations to figure that part out.

            The finish line is getting closer to having this site upgraded :-)
            Painter in training

            Comment

            • Kevin P
              Member
              • Aug 2000
              • 10809

              #7
              I noticed yesterday and today that DNS was in flux... I lost the ability to reach the site last night and was going to send you an email, but then I reset my caching DNS server (BIND9 on my Ubuntu firewall) and it started working again. Then tonight I was having trouble reaching Squirrelmail on mail.htguide.com, apparently when you set up the new DNS there's no A record or CNAME for mail. I just changed my bookmark to use www.htguide.com instead of mail.htguide.com. But once you get the email going to its new place, I won't need to check the forum email anymore anyway.

              Speaking of Ubuntu, I'm starting the process of upgrading my 3 Linux boxes at home from 18.04 to 22.04. I have the firewall install in test in a VM at the moment and will port to the actual hardware once I have everything set up. I generally prefer to do fresh installs instead of upgrading in place.

              My MythTV HTPC, which is running on hardware from 2010 and is still on 16.04 is slated to be upgraded too. I think I'll grab a mini PC on Prime Day to do that and put 22.04 and the latest MythTV on that.

              Comment

              • theSven
                Master of None
                • Jan 2014
                • 855

                #8
                Kevin P How many registrations were you seeing a day? I think the spam protections with the integrations VB has is working... I have only seen one new enrollment and one user got flagged for possible spam in a post today that I had to approve.

                One thing I have been testing today and late yesterday is converting the DB collation to utf8mb4_unicode_520_ci and character set utf8mb4. This will allow emoji's to work with mobile devices. The text editor now has an emoji icon where the links, photo, and video attachments are located. Testing on the test instance has proved successful in the conversion.
                Painter in training

                Comment

                • Kevin P
                  Member
                  • Aug 2000
                  • 10809

                  #9
                  Originally posted by svenarajala
                  Kevin P How many registrations were you seeing a day? I think the spam protections with the integrations VB has is working... I have only seen one new enrollment and one user got flagged for possible spam in a post today that I had to approve.
                  Sometimes 2-3 a day, sometimes none for several days. That is real registrations... bots would hammer the site constantly, and maybe a half dozen to a dozen blocked registration attempts per day. If you look in the user DB at users in the "Spammers Dungeon" usergroup you can see how many there were. I also used ZB Block to stop a lot of the bots so many of them couldn't even reach the registration page.

                  Are there logs in the admin side that show blocked registrations? Does it integrate with Stop Forum Spam or does it use something else?

                  That's cool you'll be enabling emojis. We have a large number of emoticons (smilies in vBulletin parlance) set up, but emojis would allow more.

                  Oh... looks like I can still do some moderator type things like approve/unapprove posts. Did you remove all my permissions? Also somewhat related: In the forum list I can still see the moderator-only areas, but I get "no permission" when I click them. Shouldn't those areas be hidden for members with no access?

                  Comment

                  • theSven
                    Master of None
                    • Jan 2014
                    • 855

                    #10
                    Originally posted by Kevin P
                    Sometimes 2-3 a day, sometimes none for several days. That is real registrations... bots would hammer the site constantly, and maybe a half dozen to a dozen blocked registration attempts per day. If you look in the user DB at users in the "Spammers Dungeon" usergroup you can see how many there were. I also used ZB Block to stop a lot of the bots so many of them couldn't even reach the registration page.

                    Are there logs in the admin side that show blocked registrations? Does it integrate with Stop Forum Spam or does it use something else?

                    That's cool you'll be enabling emojis. We have a large number of emoticons (smilies in vBulletin parlance) set up, but emojis would allow more.

                    Oh... looks like I can still do some moderator type things like approve/unapprove posts. Did you remove all my permissions? Also somewhat related: In the forum list I can still see the moderator-only areas, but I get "no permission" when I click them. Shouldn't those areas be hidden for members with no access?
                    More things to investigate!

                    I have the Stop Forum Spam and akismet enabled. I don't see logs for them though, but I guess they are working... I may not be looking in the right place yet. The ZB Block isn't used anymore, as I wasn't sure if that would work with VB5, and the on the VB support site they recommend the two spam blockers and recaptcha for a solid spam solution. I put you in the registered group, and I will double check there are no secondary groups... I thought I had unchecked them but will double check.
                    Painter in training

                    Comment

                    • Kevin P
                      Member
                      • Aug 2000
                      • 10809

                      #11
                      For some reason I was logged out this morning and couldn't log back in without resetting my password.

                      I still see moderation options. Maybe it's user-level rather than usergroup level permissions, or they're forum level permissions. Since I was an admin before some of those "god settings" might still be assigned to me.

                      ZB Block is deprecated and there are forks and something new called CIDRAM which blocks CIDRs that are toxic. Something to look into. They are PHP based and should work with any site that uses PHP. It's just inserted as an include at the top of a global page so it's checked on every call.

                      Comment

                      • masterofnone
                        HTGuide Maintainer
                        • Jul 2022
                        • 4

                        #12
                        Originally posted by Kevin P
                        For some reason I was logged out this morning and couldn't log back in without resetting my password.

                        I still see moderation options. Maybe it's user-level rather than usergroup level permissions, or they're forum level permissions. Since I was an admin before some of those "god settings" might still be assigned to me.

                        ZB Block is deprecated and there are forks and something new called CIDRAM which blocks CIDRs that are toxic. Something to look into. They are PHP based and should work with any site that uses PHP. It's just inserted as an include at the top of a global page so it's checked on every call.
                        Can you share a screenshot of what you're seeing for the moderation? I made sure you are only registered and in no other group that has moderator. If you don't want to attach to the forum send me an email 😀. I'm not sure what would have caused a password reset, I haven't enforced the setting for the lifetime. I did enforce the password length and require a cap, lowercase, number, and special character. I consider that a best practice. I have also enabled MFA that you can enable to use google or Microsoft authenticator to get the One Time Password (OTP).

                        Stop Forum Spam does the IP checking for the spam bots. I was trying to keep customization of the files to a minimum to make upgrades a copy paste exercise. So far there have been on registrations that I have found. So far so good, but I do want to stay vigilant as spam is out of control.

                        Comment

                        • Kevin P
                          Member
                          • Aug 2000
                          • 10809

                          #13
                          Originally posted by masterofnone

                          Can you share a screenshot of what you're seeing for the moderation?
                          Sure.

                          I see this when logged in as Kevin P. The checkboxes on each post and the moderator pop-up menu, which I have popped up here. I also see Mod Chalet on the forum list.

                          Click image for larger version

Name:	htgmod1.jpg
Views:	155
Size:	86.3 KB
ID:	925925
                          I also have an account called "testuser" which is a regular member, which I used for testing things when I was an admin. I reset the password on that user and logged in, and I don't see the moderation options. So there's still some powers wielded to my Kevin P account that aren't on a regular member that always was. Also I don't see the Mod Chalet listed on the forum list.

                          Testuser screenshot, no moderation options:
                          Click image for larger version

Name:	htgmod2.jpg
Views:	165
Size:	103.0 KB
ID:	925926
                          As I recall, there are "forum permissions" for each user, if you edit a user in Admin CP and then click the dropdown, I think there's Edit Forum Permissions as an option. Check there.

                          Comment

                          • theSven
                            Master of None
                            • Jan 2014
                            • 855

                            #14
                            OK, I haven't looked in that spot yet, let me know see what is over there. Thanks for the fast reply!
                            Painter in training

                            Comment

                            • theSven
                              Master of None
                              • Jan 2014
                              • 855

                              #15
                              Kevin P Did you want to moderate for HTPC still? I found the permissions where you said to look.
                              Painter in training

                              Comment

                              • Kevin P
                                Member
                                • Aug 2000
                                • 10809

                                #16
                                The HTPC area is pretty much dead, no point in having me moderate it. Just make me a regular non moderator member. Thanks.

                                One more thing that's missing: there used to be a "Mark all as read" option when doing a search on new topics/posts. I would use that to clear the new posts search after I'd read the ones I was interested in. Now I have to click on every new post to clear it from the search. That would also fix the issue some have had seeing old posts in the "new topics" search.

                                Comment

                                • theSven
                                  Master of None
                                  • Jan 2014
                                  • 855

                                  #17
                                  Originally posted by Kevin P
                                  The HTPC area is pretty much dead, no point in having me moderate it. Just make me a regular non moderator member. Thanks.

                                  One more thing that's missing: there used to be a "Mark all as read" option when doing a search on new topics/posts. I would use that to clear the new posts search after I'd read the ones I was interested in. Now I have to click on every new post to clear it from the search. That would also fix the issue some have had seeing old posts in the "new topics" search.
                                  I will get you all converted, and thanks for all your help!

                                  I didn't remove that setting that would allow the read all. I have to inquire about that setting.
                                  Painter in training

                                  Comment

                                  • theSven
                                    Master of None
                                    • Jan 2014
                                    • 855

                                    #18
                                    From the looks of it, the marked all as read was removed in VB5.
                                    Painter in training

                                    Comment

                                    • Kevin P
                                      Member
                                      • Aug 2000
                                      • 10809

                                      #19
                                      Looks like my mod powers are gone now, and Mod Chalet no longer shows up. Looks like my transformation to mere mortal is complete. Maybe check the other former admins/mods to make sure their accounts are fixed up the same, unless you still want them to be admins/mods.

                                      Maybe there's an add-on to make "mark all read" available again? Without it, threads with new posts I have no interest in will show up in my new topics search forever.

                                      Comment

                                      • theSven
                                        Master of None
                                        • Jan 2014
                                        • 855

                                        #20
                                        Originally posted by Kevin P
                                        Looks like my mod powers are gone now, and Mod Chalet no longer shows up. Looks like my transformation to mere mortal is complete. Maybe check the other former admins/mods to make sure their accounts are fixed up the same, unless you still want them to be admins/mods.

                                        Maybe there's an add-on to make "mark all read" available again? Without it, threads with new posts I have no interest in will show up in my new topics search forever.
                                        I found a setting that should mark everything after 5 days read if you haven't logged for over a week. I need to look for an add-on. I'm headed out on vacation tomorrow, and if I get any spare time I will try to look for any options.
                                        Painter in training

                                        Comment

                                        • Kevin P
                                          Member
                                          • Aug 2000
                                          • 10809

                                          #21
                                          I was logged out again this morning, but I was able to log back in without having to reset my password this time. Does the login cookie expire after a week or something?

                                          Enjoy your vacation.

                                          Comment

                                          • theSven
                                            Master of None
                                            • Jan 2014
                                            • 855

                                            #22
                                            Originally posted by Kevin P
                                            I was logged out again this morning, but I was able to log back in without having to reset my password this time. Does the login cookie expire after a week or something?

                                            Enjoy your vacation.
                                            I have to check on the login cookie. I never changed that from the migration, and will search on that to see where it is set and what options I have to adjust it.

                                            Thanks on the vacation, this is my first attempt this year to try and take time off. Looking forward to photography and getting outside.
                                            Painter in training

                                            Comment

                                            • theSven
                                              Master of None
                                              • Jan 2014
                                              • 855

                                              #23
                                              According to vBulletin the cookie should be yearly...

                                              This link from vBulletin has good info for me to investigate on the cookies. Will look into that and see if I have that configured right then...
                                              Painter in training

                                              Comment

                                              • Kevin P
                                                Member
                                                • Aug 2000
                                                • 10809

                                                #24
                                                I was looking at my stored cookies for htguide.com and found there were duplicates, perhaps from the old site as well as the new one. Expiration dates ranged from end of July 2023 to 8/7/2023. There were also cookies under both htguide.com and www.htguide.com so I deleted all of them and logged in again. The new cookies have 8/7/2023 expiration dates and exist only under www.htguide.com. Maybe one of the old cookies was passed which confused the site and it logged me out. With a "fresh batch" hopefully I'll stay logged in now.

                                                If I get logged out again I'll let you know. But, for others with the same issue, I'd clear ALL cookies stored in your browser under htguide.com and www.htguide.com and then log in again (and click Remember Me).

                                                Comment

                                                • theSven
                                                  Master of None
                                                  • Jan 2014
                                                  • 855

                                                  #25
                                                  Test Uploading an attachment to a post
                                                  Click image for larger version

Name:	svenHTGuide.png
Views:	140
Size:	71.9 KB
ID:	926148
                                                  Painter in training

                                                  Comment

                                                  Working...
                                                  😀
                                                  😂
                                                  🥰
                                                  😘
                                                  🤢
                                                  😎
                                                  😞
                                                  😡
                                                  👍
                                                  👎
                                                  Searching...Please wait.
                                                  An unexpected error was returned: 'Your submission could not be processed because you have logged in since the previous page was loaded.

                                                  Please push the back button and reload the previous window.'
                                                  An unexpected error was returned: 'Your submission could not be processed because the token has expired.

                                                  Please push the back button and reload the previous window.'
                                                  An internal error has occurred and the module cannot be displayed.
                                                  There are no results that meet this criteria.
                                                  Search Result for "|||"